Polarized radar image super-resolution method, device and equipment based on scene understanding

Abstract

The invention relates to a polarized radar image super-resolution method, device and equipment based on scene understanding. The method comprises the following steps: processing a polarized radar image to obtain a total anchor point set, anchor point identification depth implicit features and a classification result of each anchor point; taking the anchor points as graph nodes and filtering; performing non-maximum suppression on the screened graph nodes to obtain an optimal graph node set and an adjacent graph node set, constructing a local topological graph based on the optimal graph node set and the adjacent graph node set, constructing a global topological graph based on the optimal graph node set, and calculating local graph features and global graph features of the optimal graph nodes; obtaining a super-resolution depth implicit feature, and performing feature fusion according to the local graph feature and the global graph feature of the optimal graph node, the pixel component identification depth implicit feature and the super-resolution depth implicit feature to obtain a fusion feature; and super-resolution decoding and re-optimization are carried out on the fusion features to obtain a super-resolution polarized radar image. By adopting the method, the super-resolution reconstruction precision and accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the field of radar imaging remote sensing technology, and in particular to a method, device and equipment for super-resolution of polarization radar images based on scene understanding. Background Art

[0002] Polarimetric radar can obtain multi-polarization scattering information of the target, which is beneficial to the interpretation of the target scattering mechanism and the inversion of characteristic parameters. As a typical imaging radar, Inverse Synthetic Aperture Radar (ISAR) can work all day and all weather, and plays an important role in the observation and monitoring of space targets.

[0003] The diverse scattering of components, uneven distribution of scattering intensity, and significant differences in texture characteristics in ISAR images of space targets pose challenges to fine-grained reconstruction. It is noted that there is a correlation between spatial positions and categories between components, that is, the texture characteristics of components of the same type are similar, and the textures of adjacent components are correlated. Utilizing this correlation is expected to improve the performance of super-resolution reconstruction.

[0004] Most existing super-resolution reconstruction methods are aimed at optical images, using convolutional neural networks for pixel-level end-to-end processing. They cannot fully extract the overall characteristics of components and the associated features between components, and lose important information that is helpful for super-resolution reconstruction, affecting the refined reconstruction of the target. For the processing of target relationships, deep learning-based methods generally consider graph neural networks, and mainly mine object relationships by constructing K-nearest neighbor graphs. In the way of constructing graph structures, the relationship between local graph nodes and global graph nodes is less considered. The model cannot well understand the position and role of the target in the overall scene, as well as the relationship between local features and overall structures, affecting the overall quality and accuracy of the reconstructed image. On the other hand, for radar images, targets often present sparse characteristics and only occupy part of the image area. When constructing the graph structure of radar images, traditional methods also include a large number of background nodes that are not related to the target in the graph structure construction, which increases the amount of calculation and the complexity of the model, and affects the effect of super-resolution reconstruction. Summary of the invention

[0005] Based on this, it is necessary to provide a polarization radar image super-resolution method, device and equipment based on scene understanding to address the above technical problems.

[0006] A polarimetric radar image super-resolution method based on scene understanding, the method comprising:

[0007] Obtain a polarimetric radar image, input the polarimetric radar image into a pre-trained anchor-based deep recognition network, and obtain a total anchor point set, anchor point recognition deep implicit features of each anchor point in the total anchor point set, and classification results;

[0008] Taking the anchor points in the total anchor point set as graph nodes, performing filtering processing on each graph node to obtain filtered graph nodes;

[0009] Perform non-maximum suppression on the screened graph nodes, obtain the optimal graph node set and the adjacent graph node set based on the non-maximum suppression result, construct a local topology graph according to the correlation between the nodes in the optimal graph node set and the adjacent graph node set, process the anchor point recognition deep implicit features according to the preset local graph weighting coefficient, and obtain the local graph features of the optimal graph node in the local topology graph;

[0010] Constructing a global topology map according to the correlation between the optimal graph nodes, processing the anchor point recognition deep implicit features according to the pre-set global graph weighting coefficients, and obtaining the global graph features of the optimal graph nodes in the global topology map;

[0011] Obtaining a super-resolution deep implicit feature of each pixel in the polarimetric radar image, performing feature fusion according to the local graph feature of the optimal graph node, the global graph feature, the pixel component recognition deep implicit feature and the super-resolution deep implicit feature, to obtain a fusion feature; the pixel component recognition deep implicit feature is obtained by the anchor point recognition deep implicit feature;

[0012] The fusion features are super-resolution decoded to obtain a super-resolution reconstruction result, and the super-resolution reconstruction result is optimized to obtain a super-resolution polarimetric radar image.

[0013] A polarization radar image super-resolution device based on scene understanding, the device comprising:

[0014] An anchor point recognition module is used to obtain polarimetric radar images, input the polarimetric radar images into a pre-trained anchor point-based deep recognition network, and obtain the total anchor point set, the anchor point recognition deep implicit features of each anchor point in the total anchor point set, and the classification results;

[0015] A graph node filtering module, used to use the anchor points in the total anchor point set as graph nodes, perform filtering processing on each graph node, and obtain filtered graph nodes;

[0016] A local graph feature extraction module is used to perform non-maximum suppression on the screened graph nodes, obtain an optimal graph node set and a neighboring graph node set based on the non-maximum suppression result, construct a local topology graph according to the correlation between the nodes in the optimal graph node set and the neighboring graph node set, and calculate the local graph features of the optimal graph node in the local topology graph according to a preset local graph weighting coefficient;

[0017] A global graph feature extraction module, used to construct a global topology graph according to the correlation between the optimal graph nodes, and calculate the global graph features of the optimal graph nodes in the global topology graph according to a preset global graph weighting coefficient;

[0018] A feature fusion module, used to obtain the super-resolution deep implicit feature of each pixel in the polarimetric radar image, and perform feature fusion according to the local graph feature of the optimal graph node, the global graph feature, the pixel component recognition deep implicit feature and the super-resolution deep implicit feature to obtain a fused feature; the pixel component recognition deep implicit feature is obtained by the anchor point recognition deep implicit feature;

[0019] The super-resolution module is used to perform super-resolution decoding on the fusion features to obtain a super-resolution reconstruction result, and optimize the super-resolution reconstruction result to obtain a super-resolution polarimetric radar image.

[0020] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0021] Obtain a polarimetric radar image, input the polarimetric radar image into a pre-trained anchor-based deep recognition network, and obtain a total anchor point set, anchor point recognition deep implicit features of each anchor point in the total anchor point set, and classification results;

[0022] Taking the anchor points in the total anchor point set as graph nodes, performing filtering processing on each graph node to obtain filtered graph nodes;

[0023] Perform non-maximum suppression on the screened graph nodes, obtain the optimal graph node set and the adjacent graph node set based on the non-maximum suppression result, construct a local topology graph according to the correlation between the nodes in the optimal graph node set and the adjacent graph node set, process the anchor point recognition deep implicit features according to the preset local graph weighting coefficient, and obtain the local graph features of the optimal graph node in the local topology graph;

[0024] Constructing a global topology map according to the correlation between the optimal graph nodes, processing the anchor point recognition deep implicit features according to the pre-set global graph weighting coefficients, and obtaining the global graph features of the optimal graph nodes in the global topology map;

[0025] Obtaining a super-resolution deep implicit feature of each pixel in the polarimetric radar image, performing feature fusion according to the local graph feature of the optimal graph node, the global graph feature, the pixel component recognition deep implicit feature and the super-resolution deep implicit feature, to obtain a fusion feature; the pixel component recognition deep implicit feature is obtained by the anchor point recognition deep implicit feature;

[0026] The fusion features are super-resolution decoded to obtain a super-resolution reconstruction result, and the super-resolution reconstruction result is optimized to obtain a super-resolution polarimetric radar image.

[0027] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0028] Obtain a polarimetric radar image, input the polarimetric radar image into a pre-trained anchor-based deep recognition network, and obtain a total anchor point set, anchor point recognition deep implicit features of each anchor point in the total anchor point set, and classification results;

[0029] Taking the anchor points in the total anchor point set as graph nodes, performing filtering processing on each graph node to obtain filtered graph nodes;

[0030] Perform non-maximum suppression on the screened graph nodes, obtain the optimal graph node set and the adjacent graph node set based on the non-maximum suppression result, construct a local topology graph according to the correlation between the nodes in the optimal graph node set and the adjacent graph node set, process the anchor point recognition deep implicit features according to the preset local graph weighting coefficient, and obtain the local graph features of the optimal graph node in the local topology graph;

[0031] Constructing a global topology map according to the correlation between the optimal graph nodes, processing the anchor point recognition deep implicit features according to the pre-set global graph weighting coefficients, and obtaining the global graph features of the optimal graph nodes in the global topology map;

[0032] Obtaining a super-resolution deep implicit feature of each pixel in the polarimetric radar image, performing feature fusion according to the local graph feature of the optimal graph node, the global graph feature, the pixel component recognition deep implicit feature and the super-resolution deep implicit feature, to obtain a fusion feature; the pixel component recognition deep implicit feature is obtained by the anchor point recognition deep implicit feature;

[0033] The fusion features are super-resolution decoded to obtain a super-resolution reconstruction result, and the super-resolution reconstruction result is optimized to obtain a super-resolution polarimetric radar image.

[0034] The above-mentioned scene-understanding-based polarimetric radar image super-resolution method, device and equipment, by inputting the polarimetric radar image into a pre-trained anchor-based deep recognition network, obtains the total anchor set, the anchor recognition deep implicit features of each anchor point in the total anchor set and the classification results, can guide the subsequent super-resolution reconstruction, filter the graph nodes, can screen out the target-related graph nodes, reduce the amount of calculation and complexity, improve the processing speed, perform non-maximum suppression on the screened graph nodes and construct a local topology map, can capture the detailed features and associated information of the local area components, improve the reconstruction accuracy, and by constructing a global topology map, can integrate global context information, fuse multiple features, realize differentiated super-resolution reconstruction, and improve the image resolution and quality through super-resolution decoding and optimization. The embodiment of the present invention can improve the precision and accuracy of super-resolution reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1A schematic diagram of a process of a polarization radar image super-resolution method based on scene understanding in one embodiment;

[0036] Figure 2 A schematic diagram of a process for obtaining high-resolution polarimetric radar image data in a specific embodiment;

[0037] Figure 3 is a structural block diagram of a polarization radar image super-resolution device based on scene understanding in one embodiment;

[0038] Figure 4 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0040] In one embodiment, Figure 1 As shown, a polarization radar image super-resolution method based on scene understanding is provided, comprising the following steps:

[0041] Step 102, obtaining a polarimetric radar image, inputting the polarimetric radar image into a pre-trained anchor-based deep recognition network, and obtaining a total anchor point set, anchor point recognition deep implicit features of each anchor point in the total anchor point set, and a classification result.

[0042] The polarimetric radar image may be a dual-channel polarimetric radar image, a polarimetric interferometric SAR image, a dual-polarimetric SAR image, a full-polarimetric SAR image, or other polarimetric radar image related fields. The anchor-based deep recognition network may use a YOLO series model, etc.

[0043] Anchors serve as initial position markers where the target may be located, providing basic positioning for subsequent operations. The anchor recognition deep implicit features contain rich anchor feature information, and the anchor classification results can determine the component category. This information guides the subsequent analysis and processing of the components, thereby guiding the super-resolution reconstruction process. For example, anchor classification determines that a certain area may be a specific component, and subsequent processing revolves around this judgment, gradually building a feature representation related to the component for reconstruction.

[0044] Step 104 , taking the anchor points in the total anchor point set as graph nodes, performing filtering processing on each graph node to obtain filtered graph nodes.

[0045] By filtering the initial anchor point as a graph node, we can filter out the target-related graph nodes. It can be understood that filtering effectively reduces the amount of calculation and the complexity of the model, so that subsequent computing resources can be more concentrated on processing information related to the target component, thereby improving the processing speed. At the same time, after removing background interference, the model can focus on the target component more accurately.

[0046] Step 106, perform non-maximum suppression on the filtered graph nodes, obtain the optimal graph node set and the neighboring graph node set based on the non-maximum suppression results, construct a local topology graph according to the correlation between the nodes in the optimal graph node set and the neighboring graph node set, process the anchor point recognition deep implicit features according to the pre-set local graph weighting coefficients, and obtain the local graph features of the optimal graph nodes in the local topology graph.

[0047] By performing non-maximum suppression on the screened graph nodes, the most representative nodes, i.e., the optimal graph nodes, can be retained. After obtaining the local optimal graph node set, the eliminated graph nodes (i.e., nodes with a high degree of overlap with the optimal graph nodes) constitute the set of neighboring graph nodes. Although these nodes are not optimal in the local area, they have a certain spatial correlation with the optimal graph nodes. By connecting graph nodes of the same category and with high spatial correlation (IOU values ​​meet a certain threshold), a local topology graph is constructed. The nodes in the local topology graph represent rotation boxes, and the edges represent the connection relationship between the optimal graph nodes and the adjacent neighboring graph nodes. In this way, graph nodes with similar semantic information and close spatial positions can be associated, so that the network can better utilize this spatial correlation information and make more accurate judgments and inferences on the location and category of the target. For example, when detecting multiple targets of the same type, the graph nodes corresponding to these targets can form an association structure through connection, which helps to comprehensively judge the overall situation of the target.

[0048] By calculating local image features, it is possible to capture the detailed features of the components in the local area and the correlation information between them, providing rich local detail information for super-resolution reconstruction. This local information is crucial for accurately restoring the texture and structure of the components, helping to improve the reconstruction accuracy and make the texture structure of the reconstruction result more refined.

[0049] Step 108, constructing a global topology map according to the correlation between the optimal graph nodes, processing the anchor point recognition deep implicit features according to the preset global graph weighting coefficients, and obtaining the global graph features of the optimal graph nodes in the global topology map.

[0050] By building a global topology map based on the relationship between the optimal graph nodes, the information of each local area is integrated. The nodes in the global topology map represent rotation boxes, and the edges represent the connection relationship between the optimal graph nodes and the adjacent optimal graph nodes. This can help the model better understand the position and role of the target in the overall scene and incorporate global context information into the super-resolution reconstruction process. The global graph features contain the overall structure and layout information of the target, which complement the local graph features, further improving the reconstruction accuracy and making the reconstruction results more consistent with the real structure of the target as a whole.

[0051] Step 110, obtaining the super-resolution deep implicit features of each pixel in the polarimetric radar image, performing feature fusion according to the local graph features of the optimal graph node, the global graph features, the pixel component recognition deep implicit features and the super-resolution deep implicit features, and obtaining fused features.

[0052] The pixel component recognition deep implicit features are obtained through the anchor point recognition deep implicit features. The pixel component recognition deep implicit features reflect the component semantics, and the super-resolution deep implicit features reflect the pixel-level details. By fusing multiple features, the similarity of the texture characteristics of the same components and the relationship between different components in spatial position and semantics are fully utilized. According to the texture conditions of different components, the fused features can provide richer and more accurate information for super-resolution reconstruction, thereby achieving differentiated super-resolution reconstruction and significantly improving the reconstruction accuracy.

[0053] Step 112, super-resolution decoding is performed on the fused features to obtain a super-resolution reconstruction result, and the super-resolution reconstruction result is optimized to obtain a super-resolution polarimetric radar image.

[0054] Through super-resolution decoding, the fused features are converted into super-resolution reconstructed images, which initially improves the image resolution. After the optimization process, the clarity and quality of the image are further improved. Finally, a super-resolution polarimetric radar image is obtained, which greatly improves the accuracy of super-resolution reconstruction.

[0055] In the above-mentioned polarimetric radar image super-resolution method based on scene understanding, by inputting the polarimetric radar image into the pre-trained anchor-based deep recognition network, the total anchor set, the anchor recognition deep implicit features of each anchor in the total anchor set and the classification results are obtained, which can guide the subsequent super-resolution reconstruction, filter the graph nodes, filter out the target-related graph nodes, reduce the amount of calculation and complexity, and improve the processing speed. The non-maximum suppression of the filtered graph nodes and the construction of the local topology map can capture the detailed features and related information of the local area components, improve the reconstruction accuracy, and by constructing the global topology map, it can integrate the global context information, fuse multiple features, realize differentiated super-resolution reconstruction, and improve the image resolution and quality through super-resolution decoding and optimization. The embodiment of the present invention can improve the precision and accuracy of super-resolution reconstruction.

[0056] In one embodiment, a polarimetric radar image is input into a pre-trained anchor-based deep recognition network to obtain a total anchor set, anchor recognition deep implicit features of each anchor in the total anchor set, and a classification result, including: inputting the polarimetric radar image into a pre-trained anchor-based deep recognition network; the anchor-based deep recognition network includes a backbone network, a neck network, and a recognition head; the backbone network includes a plurality of cascaded CSP modules; the neck network includes a progressive asymmetric feature pyramid module; the intermediate implicit features of the polarimetric radar image are extracted step by step through the CSP modules of the backbone network; the neck network is used to extract the intermediate implicit features of the polarimetric radar image according to the backbone network; The intermediate implicit features output by the last three CSP modules of the network are fused to obtain anchor feature maps corresponding to multiple levels; the position parameters and category parameters of the anchors in the anchor feature maps of multiple levels are extracted through the recognition head; the anchors are sampled in the spatial domain; the total anchor set is obtained according to the anchor sets corresponding to the anchors of multiple levels, and the classification results of the anchors are obtained according to the category parameters of the anchors in the total anchor set; the anchor feature maps corresponding to multiple levels are cascaded and spliced ​​to obtain the anchor recognition deep implicit feature set; the anchor recognition deep implicit feature set includes the anchor recognition deep implicit features of each anchor in the total anchor set.

[0057] Specifically, taking a dual-channel polarization radar as an example, the polarization radar scattering vector data obtained is V = [S LL ,S RL ] T The YOLO algorithm is used as the basic recognizer, and the CSP module with a convolution kernel size of 5×5 is used to build a backbone network model with a 5-layer structure, which is marked with P1-P5 respectively. P1-P5 are connected in a cascade manner, and the spatial dimension of the feature map is gradually reduced. The progressive asymmetric feature pyramid module is used to build the neck network model, which is used to fuse the intermediate implicit features output by the P3, P4 and P5 layers of the backbone network model to obtain a multi-level anchor feature set {f levelL}, L = 1, 2, 3. For a polarimetric radar image of H×W pixel size, S T Sampling to obtain three levels of anchor point sets Among them, N SP is the number of anchor points. Each anchor point contains the absolute position feature {x J ,y J} L Then, the constructed recognition head 1 is used to extract the position and category parameters of each anchor point. For each level of the feature f obtained by the neck network module levelL , using a recognition block, including two stacked convolution blocks and three 1×1 convolution layers. Each convolution block consists of a 3×3 convolution layer, a batch normalization layer (Batch Norm, BN) layer and a SiLU activation layer. The three 1×1 convolution layers are used to extract the offset position vector {x p,y p} and size vector {h,w}, angle parameter θ and category score vector ζ. The position vector {x,y} of the component is calculated as the sum of the absolute position vector and the offset position vector, which is

[0058] {x,y}={x p ,y p}+{x J ,y J}(1)

[0059] Note that cls is the category corresponding to the maximum value in the score vector ζ of category A. In order to uniformly analyze the relationship between anchor features at different levels, the anchor feature maps f levelL Cascade splicing is performed to obtain the anchor point recognition deep implicit feature set f all ,for

[0060] f all =f level1 ||f level2 ||f level3 (2)

[0061] Among them, || represents the feature concatenation operation. The anchor point sets of the three levels are merged to construct the total anchor point set {SP} all .

[0062] In one embodiment, each graph node is filtered to obtain a filtered graph node, including: extracting multiple polarization features of the polarization radar image, extracting a target binary mask according to each polarization feature, and obtaining a corresponding binary mask feature map; the multiple polarization features include horizontal-horizontal polarization features, vertical-horizontal polarization features, and difference polarization features; fusing the binary mask feature map to obtain a fused mask, and obtaining the filtered graph node according to the anchor point in the area with a value of 1 in the fused mask.

[0063] In this embodiment, the total anchor point set {SP} all Each anchor point in is regarded as a graph node, and a graph node set is established in, Represents a graph node, and A is the number of graph nodes. Extract three polarization features of the polarization radar image: horizontal-horizontal polarization feature, vertical-horizontal polarization feature, and difference polarization feature, i.e., amplitude feature |S LL |、|S RL | and its difference |S LL -S RL |. Extract the target binary mask for each of these three feature maps and Use the logical OR operation | to fuse these three binary mask feature maps to get the fused mask for

[0064]

[0065] Anchor points outside the value 1 area are not considered, and the filtered graph node set is obtained.

[0066] In one embodiment, obtaining a super-resolution deep implicit feature for each pixel in a polarimetric radar image includes: obtaining polarimetric radar scattering vector data of the polarimetric radar image, rearranging the polarimetric radar scattering vector data to obtain input data, and inputting the input data into a pre-trained super-resolution coding network based on a dense residual structure to obtain a super-resolution deep implicit feature for each pixel in the polarimetric radar image.

[0067] Specifically, a super-resolution coding network based on a dense residual structure is constructed to transform the complex polarization radar scattering vector data V = [S LL ,S RL ] T Rearranged to input data I,

[0068] I={[Re(S LL ),Im(S LL ),Re(S RL ),Im(S RL )]} T (4)

[0069] Input I into the super-resolution encoding module based on the dense residual structure to obtain the super-resolution deep implicit feature f of size H×W×64 super The specific implementation method is recorded in Chinese patent CN114972041A "Polarization radar image super-resolution reconstruction method and device based on residual network", which will not be repeated here.

[0070] In one embodiment, a local topological map is constructed based on the correlation between nodes in an optimal graph node set and a neighboring graph node set, including: calculating the IOU values ​​of the optimal graph nodes in the optimal graph node set and the neighboring graph nodes in the neighboring graph node set, connecting the neighboring graph nodes and the optimal graph nodes that belong to the same category and whose IOU values ​​meet a spatial correlation threshold, and constructing a local topological map based on the connection relationship between the nodes in the optimal graph node set and the neighboring graph node set.

[0071] Specifically, after filtering the graph nodes using the multi-channel graph node filtering module, filter , use the non-maximum suppression algorithm to obtain the local optimal graph node set Θ Optimum ,for

[0072]

[0073] Other suppressed rotation boxes construct neighboring graph node sets Θ other ,for

[0074]

[0075] Where A and B are the number of graph nodes. Optimum Each graph node in Calculate its difference with Θ other Each graph node in IOU valueυ ab ,for

[0076] υ ab =F IOU (x a ,x b ) (7)

[0077] χ=F corner (x,y,h,w,θ)(8)

[0078] Where χ is the coordinate of the corner point of the rotation box. IOU (·) is the IOU calculation function. F corner (·) is the position parameter conversion function, which is used to calculate the corner coordinates from the position parameters {x, y, h, w, θ}. In actual operation, the low spatial position correlation rotation frame with υ<k is not considered. After preliminary identification, it can be obtained The classification score vector ζ a .cls a represents the category with the highest classification score. We will and Θ belonging to the same category cls and υ ≥ k other According to the experiment, k = 0.1 is taken to ensure the spatial correlation between the two rotation boxes, and the adjacency matrix Ω is used. local Record this connection relationship. and When connected, there is Ω local (a,b)=1. Therefore, the local topology can be constructed Each graph node Represents a rotation box, each edge e a,b ∈ε local Represents the connection relationship between two nodes. The adjacent nodes of graph node a are represented as Note that the constructed G local is an undirected graph, satisfying and On the other hand, the relevant aggregation operation is only applied to only and are connected, and Θother In There is no connection between them.

[0079] In one embodiment, the local graph weight coefficient is:

[0080]

[0081] in, is the local graph weight coefficient, W is the weight matrix, and are the anchor point recognition deep implicit features of nodes a, b, and c, respectively. H(·) is the operation of mapping from a multidimensional vector to a one-dimensional vector. is the set of adjacent nodes of a, a is the optimal graph node, b is the adjacent node of a,

[0082] In this embodiment, the importance of the adjacent nodes to the central node a is calculated to obtain the local graph weight coefficient. For the central node, the feature vector obtained from the preliminary recognition stage is f all . Local graph features of graph node a for

[0083]

[0084] In order to enable the neural network to perceive the topological relationship of the overall structure, a global graph G is constructed. global ,for

[0085]

[0086] Since the neighboring node information is used in the local topology graph, only the local optimal result is used in the global graph branch. Construct a global graph. Specifically, prioritize non-local nodes with higher classification score similarity and calculate the cosine distance CD between the classification score vectors of two rotation boxes. ab ,for

[0087]

[0088] The higher the CD value, the more similar the component category results estimated by the two rotation boxes are. Then, a k-nearest neighbor graph is constructed based on the top k highest CD values ​​to obtain the adjacency matrix Ω global If Ω global (a,b)=1, then the box Box is rotated in the graph structure a and Box b There is a connection relationship between them, otherwise they are not connected in the graph. For this, the global graph is constructed Similarly, G globalis an undirected graph. In constructing the global graph, the criterion for measuring the similarity between the classification score vectors of two rotating boxes can be considered, in addition to the cosine distance, similarity measurement criteria such as Euclidean distance and Manhattan distance. The calculation method of the global graph feature of the optimal graph node is similar to the calculation method of the local graph feature. The global graph weight coefficient is calculated by using the anchor point recognition deep implicit feature, and then the global graph weight coefficient is used for weighted calculation to obtain the global graph feature of the optimal graph node.

[0089] In one embodiment, the step of obtaining implicit features of pixel component recognition includes: generating a component recognition rotation box according to the classification result of the anchor point, obtaining a component mask according to the component recognition rotation box, and obtaining a pixel component recognition implicit feature of each pixel in the component mask according to the anchor point recognition depth implicit feature.

[0090] Specifically, the component mask is generated according to the kth component recognition rotation box Pixels within the widget's rotation box are assigned a value of 1, otherwise 0. Each pixel in the rotation box is assigned the anchor feature corresponding to the rotation box. Note that each pixel may correspond to multiple rotation boxes. A conversion module is constructed to perform dimension matching and process the anchor point recognition deep implicit features to obtain the pixel component recognition implicit feature f recog Specifically, the module consists of a 1×1 convolutional layer and a SiLU activation layer, with the input feature dimension being the number of components and the output feature dimension being 1.

[0091] Considering that residual learning can promote gradient propagation and deep learning network convergence, the present invention constructs a feature fusion module based on residual structure to fuse pixel components to identify implicit features f recog , super-resolution deep implicit feature f super , local graph features and global graph features Since a multi-layer decoder is used for parameter estimation in the recognition network, this work only uses the shallow convolutional block Conv(·) for feature fusion and dimension matching. Specifically, this module consists of five convolutional blocks in series, each of which contains a 3×3 convolutional layer and a SiLU activation layer. The fused and enhanced features for

[0092]

[0093] In a specific embodiment, Figure 2 As shown, a schematic diagram of a process for obtaining high-resolution polarimetric radar image data is provided, which specifically includes the following steps:

[0094] (S1) constructing an anchor-based component recognition module to perform component recognition and extract deep implicit features for anchor recognition;

[0095] (S2) treating anchor points as graph nodes and establishing a graph node set;

[0096] (S3) constructing a graph node filtering module to process the graph node set;

[0097] (S4) constructing a super-resolution encoding module based on a dense residual structure to extract super-resolution deep implicit features at each pixel position;

[0098] (S5) establishing a scene graph, constructing a dual-branch graph attention module for feature enhancement, and obtaining local features and global features;

[0099] (S6) extracting component recognition deep implicit features, and fusing the component recognition deep implicit features, super-resolution deep implicit features, local graph features, and global graph features;

[0100] (S7) constructing a super-resolution decoding module to perform super-resolution reconstruction and decoding on the fused deep features;

[0101] (S8) Use the trained network model to perform super-resolution processing on the low-resolution polarimetric radar image.

[0102] The specific implementation of steps (S1)-(S6) has been described in the above embodiments. Step (S7) constructs a super-resolution decoding module to perform super-resolution reconstruction decoding on the enhanced deep features. Specifically, the super-resolution decoding module based on a multi-layer perceptron is constructed to decode the fused and enhanced features. Decoding is performed to obtain the final super-resolution reconstruction result. Step (S8) uses the trained network model to perform super-resolution processing on the low-resolution polarimetric radar image. Specifically, according to the high-resolution true value I HR , using L1 loss function loss l1 Super-resolution reconstruction results I SR Optimize for

[0103]

[0104] in, Respectively The number of the pth channel, N is the number of channels. The cross entropy loss function is used to optimize the category parameter cls in part recognition, and the intersection over union (IOU) loss function is used to optimize the position parameters {x, y, h, w, θ} in part recognition, that is, the IOU between the estimated rotation box and the true value is calculated, and the gradient back propagation is performed to optimize the network weight parameters.

[0105] It should be understood that although Figure 1The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0106] In one embodiment, Figure 3 As shown, a polarization radar image super-resolution device based on scene understanding is provided, comprising:

[0107] An anchor point recognition module 302 is used to obtain a polarimetric radar image, input the polarimetric radar image into a pre-trained anchor point-based deep recognition network, and obtain a total anchor point set, anchor point recognition deep implicit features of each anchor point in the total anchor point set, and a classification result;

[0108] A graph node filtering module 304 is used to use the anchor points in the total anchor point set as graph nodes, perform filtering processing on each graph node, and obtain filtered graph nodes;

[0109] The local graph feature extraction module 306 is used to perform non-maximum suppression on the filtered graph nodes, obtain the optimal graph node set and the adjacent graph node set based on the non-maximum suppression result, construct a local topology graph according to the correlation between the nodes in the optimal graph node set and the adjacent graph node set, and process the anchor point recognition deep implicit features according to the preset local graph weighting coefficient to obtain the local graph features of the optimal graph node in the local topology graph;

[0110] A global graph feature extraction module 308 is used to construct a global topology graph according to the correlation between the optimal graph nodes, and process the anchor point recognition deep implicit features according to the preset global graph weighting coefficient to obtain the global graph features of the optimal graph nodes in the global topology graph;

[0111] The feature fusion module 310 is used to obtain the super-resolution deep implicit feature of each pixel in the polarimetric radar image, and perform feature fusion according to the local graph feature of the optimal graph node, the global graph feature, the pixel component recognition deep implicit feature and the super-resolution deep implicit feature to obtain a fused feature; the pixel component recognition deep implicit feature is obtained by the anchor point recognition deep implicit feature;

[0112] The super-resolution module 312 is used to perform super-resolution decoding on the fused features to obtain a super-resolution reconstruction result, and optimize the super-resolution reconstruction result to obtain a super-resolution polarimetric radar image.

[0113] In one of the embodiments, the method is also used to input the polarimetric radar image into a pre-trained anchor-based deep recognition network; the anchor-based deep recognition network includes a backbone network, a neck network and a recognition head; the backbone network includes a plurality of cascaded CSP modules; the neck network includes a progressive asymmetric feature pyramid module; the intermediate implicit features of the polarimetric radar image are extracted step by step through the CSP modules of the backbone network; the neck network is used to fuse the intermediate implicit features output by the last three CSP modules of the backbone network to obtain anchor feature maps corresponding to multiple levels; the position parameters and category parameters of the anchors in the anchor feature maps of multiple levels are extracted through the recognition head; the anchors are sampled in the spatial domain; the total anchor set is obtained according to the anchor sets corresponding to the anchors of the multiple levels, and the classification results of the anchors are obtained according to the category parameters of the anchors in the total anchor set; the anchor feature maps corresponding to the multiple levels are cascaded and spliced ​​to obtain an anchor recognition deep implicit feature set; the anchor recognition deep implicit feature set includes the anchor recognition deep implicit features of each anchor in the total anchor set.

[0114] In one of the embodiments, it is also used to extract multiple polarization features of a polarization radar image, extract a target binary mask according to each polarization feature, and obtain a corresponding binary mask feature map; the multiple polarization features include horizontal-horizontal polarization features, vertical-horizontal polarization features, and difference polarization features; the binary mask feature map is fused to obtain a fused mask, and the filtered graph nodes are obtained according to the anchor points in the area with a value of 1 in the fused mask.

[0115] In one of the embodiments, it is also used to calculate the IOU value between the optimal graph node in the optimal graph node set and the neighboring graph nodes in the neighboring graph node set, connect the neighboring graph nodes and the optimal graph nodes that belong to the same category and whose IOU values ​​meet the spatial correlation threshold, and construct a local topological graph based on the connection relationship between the nodes in the optimal graph node set and the neighboring graph node set.

[0116] In one embodiment, the local graph weight coefficient is:

[0117]

[0118] in, is the local graph weight coefficient, W is the weight matrix, and are the anchor point recognition deep implicit features of nodes a, b, and c, respectively. H(·) is the operation of mapping from a multidimensional vector to a one-dimensional vector. is the set of adjacent nodes of a, a is the optimal graph node, b is the adjacent node of a,

[0119] In one of the embodiments, it is also used to obtain polarization radar scattering vector data of the polarization radar image, rearrange the polarization radar scattering vector data to obtain input data, input the input data into a pre-trained super-resolution coding network based on a dense residual structure, and obtain super-resolution deep implicit features for each pixel in the polarization radar image.

[0120] In one of the embodiments, it is also used to guide the generation of a component recognition rotation box based on the classification result of the anchor point, obtain the component mask based on the component recognition rotation box, and obtain the pixel component recognition implicit feature of each pixel in the component mask based on the anchor point recognition depth implicit feature.

[0121] For the specific definition of the polarimetric radar image super-resolution device based on scene understanding, please refer to the definition of the polarimetric radar image super-resolution method based on scene understanding above, which will not be repeated here. Each module in the above-mentioned polarimetric radar image super-resolution device based on scene understanding can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0122] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a polarization radar image super-resolution method based on scene understanding is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0123] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0124] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in the above embodiment when executing the computer program.

[0125] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above embodiment are implemented.

[0126] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0127] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0128] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A polarimetric radar image super-resolution method based on scene understanding, characterized in that: The method comprises: Obtain a polarimetric radar image, input the polarimetric radar image into a pre-trained anchor-based deep recognition network, and obtain a total anchor point set, anchor point recognition deep implicit features of each anchor point in the total anchor point set, and classification results; Taking the anchor points in the total anchor point set as graph nodes, performing filtering processing on each graph node to obtain filtered graph nodes; Perform non-maximum suppression on the screened graph nodes, obtain the optimal graph node set and the adjacent graph node set based on the non-maximum suppression result, construct a local topology graph according to the correlation between the nodes in the optimal graph node set and the adjacent graph node set, process the anchor point recognition deep implicit features according to the preset local graph weighting coefficient, and obtain the local graph features of the optimal graph node in the local topology graph; Constructing a global topology map according to the correlation between the optimal graph nodes, processing the anchor point recognition deep implicit features according to the pre-set global graph weighting coefficients, and obtaining the global graph features of the optimal graph nodes in the global topology map; Obtaining a super-resolution deep implicit feature of each pixel in the polarimetric radar image, performing feature fusion according to the local graph feature of the optimal graph node, the global graph feature, the pixel component recognition deep implicit feature and the super-resolution deep implicit feature, to obtain a fusion feature; the pixel component recognition deep implicit feature is obtained by the anchor point recognition deep implicit feature; The fusion features are super-resolution decoded to obtain a super-resolution reconstruction result, and the super-resolution reconstruction result is optimized to obtain a super-resolution polarimetric radar image.

2. The method according to claim 1, characterized in that The polarimetric radar image is input into the pre-trained anchor-based deep recognition network to obtain the total anchor set, the anchor recognition deep implicit features of each anchor point in the total anchor set, and the classification results, including: Inputting the polarimetric radar image into a pre-trained anchor-based deep recognition network; the anchor-based deep recognition network comprises a backbone network, a neck network and a recognition head; the backbone network comprises a plurality of cascade-connected CSP modules; the neck network comprises a progressive asymmetric feature pyramid module; Extracting intermediate implicit features of the polarization radar image step by step through the CSP module of the backbone network; The neck network performs fusion processing according to the intermediate implicit features output by the last three CSP modules of the backbone network to obtain anchor feature maps corresponding to multiple levels; Extracting position parameters and category parameters of anchor points in multiple levels of anchor point feature maps through the recognition head; the anchor points are obtained by sampling in the air domain; A total anchor point set is obtained according to the anchor point sets corresponding to the anchor points of multiple levels, and a classification result of the anchor point is obtained according to the category parameters of the anchor points in the total anchor point set; The anchor feature maps corresponding to multiple levels are cascaded and spliced ​​to obtain an anchor recognition deep implicit feature set; the anchor recognition deep implicit feature set includes the anchor recognition deep implicit features of each anchor point in the total anchor point set.

3. The method according to claim 1, characterized in that Filter each graph node to obtain the filtered graph nodes, including: Extracting multiple polarization features of the polarization radar image, extracting a target binary mask according to each polarization feature, and obtaining a corresponding binary mask feature map; the multiple polarization features include horizontal-horizontal polarization features, vertical-horizontal polarization features, and difference polarization features; The binary mask feature maps are fused to obtain a fused mask, and the filtered graph nodes are obtained according to the anchor points in the area with a value of 1 in the fused mask.

4. The method according to claim 1, characterized in that: The local topology graph is constructed based on the correlation between the nodes in the optimal graph node set and the neighboring graph node set, including: The IOU values ​​of the optimal graph nodes in the optimal graph node set and the neighboring graph nodes in the neighboring graph node set are calculated, and the neighboring graph nodes and the optimal graph nodes that belong to the same category and whose IOU values ​​meet the spatial correlation threshold are connected. The local topology graph is constructed according to the connection relationship between the nodes in the optimal graph node set and the neighboring graph node set.

5. The method according to claim 1, characterized in that The local graph weight coefficient is: in, is the local graph weight coefficient, W is the weight matrix, and are the anchor point recognition deep implicit features of nodes a, b, and c, respectively. H(·) is the operation of mapping from a multidimensional vector to a one-dimensional vector. is the set of adjacent nodes of a, a is the optimal graph node, b is the adjacent node of a, 6. The method according to claim 1, characterized in that Acquiring the super-resolution depth implicit feature of each pixel in the polarization radar image includes: Polarimetric radar scattering vector data of a polarimetric radar image is obtained, the polarimetric radar scattering vector data is rearranged to obtain input data, and the input data is input into a pre-trained super-resolution coding network based on a dense residual structure to obtain a super-resolution deep implicit feature of each pixel in the polarimetric radar image.

7. The method according to claim 1, characterized in that The step of obtaining the pixel component identifying implicit features comprises: A component recognition rotation box is generated based on the classification result of the anchor point, a component mask is obtained based on the component recognition rotation box, and a pixel component recognition implicit feature of each pixel in the component mask is obtained based on the anchor point recognition deep implicit feature.

8. A polarization radar image super-resolution device based on scene understanding, characterized in that: The device comprises: An anchor point recognition module is used to obtain polarimetric radar images, input the polarimetric radar images into a pre-trained anchor point-based deep recognition network, and obtain the total anchor point set, the anchor point recognition deep implicit features of each anchor point in the total anchor point set, and the classification results; A graph node filtering module, used to use the anchor points in the total anchor point set as graph nodes, perform filtering processing on each graph node, and obtain filtered graph nodes; A local graph feature extraction module is used to perform non-maximum suppression on the screened graph nodes, obtain an optimal graph node set and a neighboring graph node set based on the non-maximum suppression result, construct a local topology graph according to the correlation between the nodes in the optimal graph node set and the neighboring graph node set, and process the anchor point recognition deep implicit features according to a preset local graph weighting coefficient to obtain the local graph features of the optimal graph node in the local topology graph; A global graph feature extraction module is used to construct a global topology graph according to the correlation between the optimal graph nodes, and process the anchor point recognition deep implicit features according to the pre-set global graph weighting coefficient to obtain the global graph features of the optimal graph nodes in the global topology graph; A feature fusion module, used to obtain the super-resolution deep implicit feature of each pixel in the polarimetric radar image, and perform feature fusion according to the local graph feature of the optimal graph node, the global graph feature, the pixel component recognition deep implicit feature and the super-resolution deep implicit feature to obtain a fused feature; the pixel component recognition deep implicit feature is obtained by the anchor point recognition deep implicit feature; The super-resolution module is used to perform super-resolution decoding on the fusion features to obtain a super-resolution reconstruction result, and optimize the super-resolution reconstruction result to obtain a super-resolution polarimetric radar image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images