Cross-source remote sensing data target recognition method based on signed distance feature
Through the cross-original remote sensing data target recognition method based on symbol distance characteristics, the problem of large differences in feature extraction of heterologous image is solved, effective identification of heterologous data and three-dimensional feature extraction are realized, and it is suitable for cross-original remote sensing data target recognition.
Patent Information
- Application Number
- CN202111534994.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-15
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-12-15
AI Technical Summary
The existing remote sensing image recognition methods have great differences in feature extraction and land object recognition in heterologous data, lack universality, and are difficult to identify across origin.
Using a cross-original remote sensing data target recognition method based on symbol distance features, the visible light, infrared, and SAR images are extracted through the symbol distance feature feature extraction network, and three-dimensional feature extraction and fusion are performed in combination with 3D-Zernike descriptors, and the SVM classifier is used for Identify.
It realizes effective extraction and recognition of heterologous image features, retains the three-dimensional structural information of the target, has rotation invariance, efficient information expression, and small redundancy, and is suitable for cross-original remote sensing data target recognition.
Smart Images

Figure CN114266967B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to three-dimensional feature extraction and type recognition of ground objects in remote sensing images, and belongs to the technical field of remote sensing image target recognition. Background Art
[0002] Remote sensing image target recognition technology has important research significance in both civil and military fields. Its purpose is to detect objects or areas of interest from static images and to distinguish, detect and locate targets in the image. Accurate target recognition also lays the foundation for further completing more complex tasks. Remote sensing image shooting sensors mainly include: visible light, infrared and radar, etc. According to the similarities and differences of their sensor information sources, remote sensing images can also be divided into two categories: one is data collected under the same information source; the other is heterogeneous data. The existing various remote sensing image recognition methods generally have poor versatility. Most methods can only be applied to the recognition of a certain type of remote sensing data. A few cross-source recognition methods are often inefficient. Different environmental conditions are suitable for different types of sensors. Visible light sensors can obtain high-resolution images in an environment with sufficient light. Relatively speaking, infrared sensors have better effects in dark and smoky environments. Therefore, the practical application of remote sensing data is often multi-source. Therefore, the cross-source recognition method of remote sensing data has important research significance.
[0003] Infrared images, SAR images, and visible light images record different attribute information of ground objects, but due to different imaging mechanisms and conditions, there are obvious representation differences between heterogeneous images, which makes the features of heterogeneous images obtained for the same scene very different. Most of the current feature extraction methods only extract features based on the attribute results of one image, which is not universal. For example, feature extraction using descriptors such as SIFT, SURF, and ORB can extract obvious feature points from high-resolution visible light images, but due to differences in sensor imaging principles, the results obtained on the corresponding infrared images and SAR images are not significant. Therefore, if the same method can be used to extract features from heterogeneous data and ensure the retention of its effective features, it will be of great significance to the cross-source field research of remote sensing images. Summary of the invention
[0004] In order to solve the problem of large differences in heterogeneous image feature extraction and object recognition in remote sensing data target recognition, the present invention proposes a cross-source remote sensing data target recognition method based on signed distance features.
[0005] The cross-source remote sensing data target recognition method based on signed distance features includes the following steps:
[0006] S1. Obtain a remote sensing image to be identified, where the remote sensing image includes at least one of a visible light image, an infrared image, and a SAR image;
[0007] S2. Input the acquired remote sensing image to be identified into the signed distance feature extraction network for signed distance feature extraction, and obtain the distance value s from the random sampling point to the target boundary; if |s|≤target boundary threshold t, it is determined that the point belongs to the surface, and the target model true value boundary point set P_ REF. ;
[0008] When the input data is only homologous data, the boundary point set at this time is taken as the final boundary point set; when the input data is heterogeneous data of the same target, the signed distance features corresponding to the visible light image, infrared image, and SAR image are fused in proportion, and then it is determined whether the point belongs to the target surface to obtain the final boundary point set;
[0009] In the process of symbol distance feature extraction, the remote sensing images are input into their respective corresponding networks, that is, the visible light image is input into the sdf-optical network, the infrared image is input into the sdf-infrared network, and the SAR image is input into the sdf-SAR network;
[0010] The sdf-optical network includes an image feature extraction module, a random sampling point feature extraction module and a decoding module. The processing process of the sdf-optical network includes the following steps:
[0011] Image feature extraction module E i : Extract features from the input visible light image to obtain feature f i ;
[0012] Random sampling point feature extraction module E c :Extract features from random sampling points to obtain features f c ;
[0013] Decoding module D: According to the feature f i and f c Obtaining the extracted signed distance feature, the signed distance feature represents the distance from the point to the target surface;
[0014] The network structures of sdf-infrared and sdf-SAR are the same as those of sdf-optical, except that the input images are different.
[0015] S3, the obtained three-dimensional point set P_ REF Perform three-dimensional feature extraction to obtain the three-dimensional feature vector X E ;
[0016] S4. The feature vector X E Recognition is performed through the classifier to obtain the recognition result.
[0017] Furthermore, the image feature extraction module E iAdopt CoTnet-18 network.
[0018] Furthermore, the random sampling point feature extraction module E c Using pointnet network.
[0019] Furthermore, the decoding module D is composed of a multi-cycle CBN module unit, and the CBN module is a conditional batch normalization module; the CBN module specifically includes the following steps:
[0020] The feature f i Input mlp multi-layer neural network to get feature f 1 , and f 1 And the feature f extracted from the random sampling point c Perform .* operation to obtain feature f 2 , and then f 2 With feature f 1 Perform addition operation to obtain the feature f of 256*N 3 ; Then f 3 Perform the maximum pooling operation to obtain the 256*1 feature f 4 Finally, convolution and sigmoid operations are performed to obtain the final 256*N feature f 5 ; Preferably, the multi-cycle CBN module is cycled 5 times to obtain very good results; the characteristic f 5 Through a fully connected layer, we get the 1*N signed distance feature f 6 , which is the signed distance feature.
[0021] Furthermore, when the signed distance features corresponding to the visible light image, the infrared image, and the SAR image are fused in proportion, the signed distance features corresponding to the visible light image, the infrared image, and the SAR image are fused in a ratio of 5:3:2.
[0022] Further, the three-dimensional point set P_ REF The process of three-dimensional feature extraction is based on 3D-Zernike descriptor.
[0023] Furthermore, the process of extracting three-dimensional features based on the 3D-Zernike descriptor includes the following steps:
[0024] First calculate the 3D-Zernike descriptor
[0025]
[0026]
[0027]
[0028] Where n∈[0,N], l∈[0,n], m∈[-l,l]; k=(nl) / 2 and must be an integer, which is the order condition of the Nth order 3D-Zernike moment; μ∈[0,k];
[0029] Each combination of {n, l, m} that satisfies the order condition corresponds to a 3D-Zernike moment of a specific order. j is the serial number of M three-dimensional points, j = 1, 2, ..., M; is the orthogonalization coefficient; (x j ,y j ,z j ) is the coordinate of the jth point; is a harmonic function, is the arithmetic mean of the harmonic function; i is the imaginary unit; C is the combination operator;
[0030] Finally, calculate all 3D-Zernike descriptors ZD nl , forming a 3DZD descriptor vector, i.e., the feature vector X E :
[0031] ZD nl =||Ω nl ||
[0032] X E ={ZD nl}
[0033] in,
[0034] Further, S4 converts the feature vector X E The process of identification through classifier is implemented based on SVM classifier.
[0035] Furthermore, in the process of obtaining the remote sensing image to be identified in S1, the entire remote sensing image is first acquired, and then the target to be identified is selected from the entire remote sensing image and cut out from the entire remote sensing image as a separate target.
[0036] Furthermore, the sdf-optical network, the sdf-infrared network and the sdf-SAR network are pre-trained, and the training processes of the sdf-optical network, the sdf-infrared network and the sdf-SAR network are the same, wherein the parameter optimization process in the sdf-optical network training includes the following steps:
[0037] Calculate the signed distance feature f obtained during training 6The loss L(θ) between the signed distance of the input target model and the true value, and the parameters are optimized accordingly;
[0038] L(θ)=|min(t,max(t,s)-min(t,max(t,d)|
[0039] Among them, L(θ) is the loss required, θ is the learning parameter of the network model; min() is the minimum operation, max() is the maximum operation, and s is the signed distance obtained by training, that is, the signed distance feature f 6 ; d is the signed distance of the true value of the target model, and t is the set target boundary threshold.
[0040] Beneficial effects:
[0041] In order to solve the problem of heterogeneous image feature extraction and object recognition in remote sensing data target recognition, the present invention proposes a cross-source remote sensing data target recognition method based on signed distance features. By introducing the signed distance function for feature extraction, the image is judged in a three-dimensional coordinate system to obtain its boundary point set, so that cross-source (visible light, infrared, SAR) remote sensing data can be feature extracted in the same network, and the problem of large differences in heterogeneous image feature extraction and object recognition can be solved. By re-extracting the three-dimensional features of the target boundary point set, equal-length feature vectors in the same coordinate system are obtained, which not only ensures the retention of effective features and fully reflects the three-dimensional structure of the target, but also has the characteristics of efficient information expression, low redundancy, good robustness, etc. while ensuring rotation invariance, which is convenient for efficient target recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 is a flow chart of specific implementation mode 1;
[0043] Figure 2 It is a schematic diagram of the overall structure of the network model of the present invention. DETAILED DESCRIPTION
[0044] Specific implementation method 1: Combination Figure 1 and Figure 2 To explain this embodiment,
[0045] The cross-source remote sensing data target recognition method based on signed distance features described in this embodiment includes the following steps:
[0046] Step 1: Obtain the entire remote sensing image, which includes visible light images, infrared images, and SAR images; in the entire remote sensing image, select the target to be identified and cut it out from the entire remote sensing image as a separate target.
[0047] Remote sensing image capture sensors mainly include: visible light, infrared and radar. According to the different working bands of the sensors, sensors working in the visible light band capture visible light, those working in the infrared band obtain infrared images, and those working in the microwave band obtain SAR images.
[0048] The shape center of the target is adjusted to the center of the image. The cropped image should contain the target. In this embodiment, the image with the target is cropped into a square image, and the target to be identified is in the image (the target is in the center of the cropped image).
[0049] Step 2: Build three signed distance feature extraction networks respectively, and use visible light image, infrared image, and SAR image samples T_ vis , T_ IR , T_ SAR And the corresponding target model T_ MOD Input the network in pairs and train it to get the network's optimal parameter model T_ PT .
[0050] In step 2, the sdf-optical network, sdf-infrared network, and sdf-SAR network are built and trained respectively to obtain the optimal network parameter model. The three networks of sdf-optical, sdf-infrared, and sdf-SAR are the mapping from the two-dimensional image to the three-dimensional boundary point set:
[0051] f(θ)≈SDF(x), x∈R 3
[0052] SDF(x)=sx∈R 3 , s∈R
[0053] Among them, f(θ) is the implicit representation of the network, θ is the learning parameter of the network model; SDF(x) is the spatial distribution function, s is the directed signed distance of x, x is a random sampling point, R 3 is the Cartesian three-dimensional coordinate system, and R is the real number field.
[0054] Taking visible light images as an example, the process of training the network to learn the parameter θ includes the following steps:
[0055] The visible light image is compared with the corresponding target model T_ MOD The true value is input into the sdf-optical network, which includes the image feature extraction module E i , Random sampling point feature extraction module E c and a decoding module D;
[0056] During training, the sdf-optical network processing includes the following steps:
[0057] Step 21: Extract the symbol distance feature using the sdf-optical network:
[0058] Image feature extraction module E i :Extract features from the input image. The image feature extraction module uses the CoTnet-18 network module to extract features from the optical image and obtain a 1x256-dimensional feature f i CoTnet mainly replaces the 3x3 convolution in the Resnet architecture with the CoT module to extract the features of two-dimensional images. CoT is a plug-and-play module that extracts image neighborhood information k through convolution. 1 , and then calculate the attention matrix M through two consecutive 1×1 convolutions, perform softmax on M and add it to k 1 Add them together to get the output features.
[0059] Random sampling point feature extraction module E c :Extract features from random sampling points. Random sampling points do not need to be input. They are composed of 2048 points randomly sampled in a unit cube. The random sampling point feature extraction module uses the pointnet network to extract features from random sampling points and obtain a 256xN-dimensional feature f c Pointnet mainly extracts the features of three-dimensional point clouds through transfer matrix and multi-layer perception network.
[0060] Decoding module D: transforms feature f i and f c Input the decoding module D to obtain the extracted signed distance feature. The signed distance feature represents the distance from the point to the target surface, that is, input a coordinate point (x, y, z) and return the distance from the point to the three-dimensional model surface. At the same time, this distance is signed. For points outside the three-dimensional model (a three-dimensional model reconstructed from the target in the image based on the signed distance), the distance is positive, while the distance inside the model is negative. The decoding module D is composed of multi-cycle CBN (Conditional Batch Normalization) module units. The CBN module is the conditional batch normalization module.
[0061] The CBN module specifically includes the following steps: i Input mlp multi-layer neural network to get feature f 1 , and f 1 And the feature f extracted from the random sampling point c Perform .* operation to obtain feature f 2 , and then f 2 With feature f 1 Perform addition operation to obtain the feature f of 256*N 3 ; Then f 3Perform the maximum pooling operation to obtain the 256*1 feature f 4 Finally, convolution and sigmoid operations are performed to obtain the final 256*N feature f 5 ; Preferably, the multi-cycle CBN module is cycled 5 times to obtain very good results; the characteristic f 5 Through a fully connected layer, we get the 1*N signed distance feature f 6 .
[0062] Step 2: Optimize parameters:
[0063] Calculate the signed distance feature f obtained during training 6 The loss L(θ) between the signed distance of the input target model and the true value is used to optimize the parameters.
[0064] L(θ)=|min(t,max(t,s)-min(t,max(t,d)|
[0065] Among them, L(θ) is the loss required, min() is the minimum operation, max() is the maximum operation, and s is the signed distance obtained by training, that is, the signed distance feature f 6 ; d is the signed distance of the true value of the target model, and t is the set target boundary threshold.
[0066] The sdf-infrared and sdf-SAR network structures are the same as the sdf-optical network structure, and the training process is also the same. The difference lies in the different input images.
[0067] Step 3: Obtain the remote sensing image to be identified according to step 1. The remote sensing image includes at least one of visible light image, infrared image, and SAR image, and perform signed distance feature extraction: Input the obtained remote sensing image to be identified into the network built in step 2 for signed distance feature extraction (the corresponding remote sensing image is input into the corresponding network, that is, the visible light image is input into sdf-optical, the infrared image is input into sdf-infrared network, and the SAR image is input into sdf-SAR), and obtain the distance value s from the random sampling point to the target boundary. Set the threshold t, |s|≤t to determine that the point belongs to the surface, thereby obtaining the true value of the target model (actually the point cloud model reconstructed according to the signed distance) boundary point set P_ REF. .
[0068] In step 3, the trained neural network is obtained through the training process of step 2. In actual application, the extraction of signed distance features is consistent with step 2, but the parameter optimization process of step 22 is no longer performed. Enter the preset parameter threshold t, when |s|≤t, it is determined that the point belongs to the surface, thus obtaining the boundary point set P_ REF. .
[0069] When the input data is only homologous data, such as visible light images or infrared images, the boundary point set at this time is taken as the final boundary point set; when the input data is heterogeneous data of the same target, since the visible light image can truly reflect the details and the details of the infrared and SAR images are not obvious, the signed distance features are fused according to the ratio of visible light: infrared: SAR = 0.5: 0.3: 0.2, and then it is determined whether the point belongs to the target surface to obtain the final boundary point set.
[0070] For example, if only visible light and infrared are used, the visible light signed distance feature is [0.1, 0.2, 0.3, 0.5], and the infrared is [0.1, 0.1, 0.1, 0.2]. They are fused according to 5:3. After fusion, the feature at this point becomes [0.1, 0.1625, 0.225, 0.3875]. The first three columns are coordinates, and the last column is the signed distance.
[0071] Step 4: 3D feature extraction: Extract the 3D point set P_ REF Perform three-dimensional feature extraction to obtain the three-dimensional feature vector X E .
[0072] Step 4: Extract three-dimensional features of the boundary point set obtained in the previous step based on the 3D-Zernike descriptor.
[0073] According to the original definition of 3D-Zernike moment, the simplified numerical calculation method is derived as follows, and the 3D-Zernike descriptor is calculated
[0074]
[0075]
[0076]
[0077] Where n∈[0,N], l∈[0,n], m∈[-l,l]; k=(nl) / 2 and must be an integer, which is the order condition of the Nth order 3D-Zernike moment; μ∈[0,k];
[0078] Each combination of {n, l, m} that satisfies the order condition corresponds to a 3D-Zernike moment of a specific order. j=1, 2, ..., M, is the sequence number of M three-dimensional points; is the orthogonalization coefficient; (x j ,y j ,z j ) is the coordinate of the jth point; is a harmonic function, is the arithmetic mean of the harmonic functions; i is the imaginary unit; C is the combination operator.
[0079] Finally, calculate all 3D-Zernike descriptors ZD nl , forming a 3DZD descriptor vector, i.e., the feature vector X E :
[0080] ZD nl =||Ω nl ||
[0081] X E ={ZD nl}
[0082] in,
[0083] Step 5: 3D feature recognition: The 3D feature vector X extracted in step 4 is E Recognition is performed through the classifier to obtain the recognition result.
[0084] In this embodiment, the three-dimensional feature vector X extracted in step 4 is E The target category is distinguished by SVM classifier to obtain the recognition result.
[0085] The present invention may also have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.
Claims
1. A cross-source remote sensing data target recognition method based on signed distance features, characterized in that: The following steps are involved: S1. Obtain a remote sensing image to be identified, where the remote sensing image includes at least one of a visible light image, an infrared image, and a SAR image; S2, inputting the obtained remote sensing image to be identified into the signed distance feature extraction network to extract the signed distance feature, and obtaining the distance value s from the random sampling point to the target boundary; If |s|≤ target boundary threshold t, the point is determined to belong to the surface, and the target model true value boundary point set P_ REF. ; When the input data is only homologous data, the boundary point set at this time is taken as the final boundary point set; when the input data is heterogeneous data of the same target, the signed distance features corresponding to the visible light image, infrared image, and SAR image are fused in proportion, and then it is determined whether the point belongs to the target surface to obtain the final boundary point set; In the process of symbol distance feature extraction, the remote sensing images are input into their respective corresponding networks, that is, the visible light image is input into the sdf-optical network, the infrared image is input into the sdf-infrared network, and the SAR image is input into the sdf-SAR network; The sdf-optical network includes an image feature extraction module, a random sampling point feature extraction module and a decoding module. The processing process of the sdf-optical network includes the following steps: Image feature extraction module E i : Extract features from the input visible light image to obtain feature f i ; Random sampling point feature extraction module E c :Extract features from random sampling points to obtain features f c ; Decoding module D: According to the feature f i and f c Obtaining the extracted signed distance feature, the signed distance feature represents the distance from the point to the target surface; The network structures of sdf-infrared and sdf-SAR are the same as those of sdf-optical, except that the input images are different. S3, the obtained three-dimensional point set P_ REF Perform three-dimensional feature extraction to obtain the three-dimensional feature vector X E ; S4. The feature vector X E Recognition is performed through the classifier to obtain the recognition result.
2. The method for cross-source remote sensing data target recognition based on signed distance features according to claim 1 is characterized in that: The image feature extraction module E i Adopt CoTnet-18 network.
3. The cross-source remote sensing data target recognition method based on signed distance features according to claim 2 is characterized in that: The random sampling point feature extraction module E c Using pointnet network.
4. The method for cross-source remote sensing data target recognition based on signed distance features according to any one of claims 1 to 3, characterized in that: The decoding module D is composed of a multi-cycle CBN module unit, and the CBN module is a conditional batch normalization module; The following steps are involved: The feature f i Input the mlp multi-layer neural network to obtain feature f1, and compare f1 with the feature f extracted from the random sampling point c Perform .* operation to obtain feature f2, then add f2 and feature f1 to obtain 256*N feature f3; then perform maximum pooling operation on f3 to obtain 256*1 feature f4, and finally perform convolution operation and sigmoid operation to obtain the final 256*N feature f5; preferably, the multi-loop CBN module can be cycled 5 times to obtain very good results; pass feature f5 through a fully connected layer to obtain 1*N signed distance feature f6, that is, signed distance feature.
5. The method for cross-source remote sensing data target recognition based on signed distance features according to claim 4 is characterized in that: When the signed distance features corresponding to the visible light image, the infrared image, and the SAR image are fused in proportion, the signed distance features corresponding to the visible light image, the infrared image, and the SAR image are fused in a ratio of 5:3:
2.
6. The method for cross-source remote sensing data target recognition based on signed distance features according to claim 5 is characterized in that: S3 obtains the three-dimensional point set P_ REF The process of three-dimensional feature extraction is based on 3D-Zernike descriptor.
7. The method for cross-source remote sensing data target recognition based on signed distance features according to claim 6, characterized in that: The process of 3D feature extraction based on 3D-Zernike descriptor includes the following steps: First calculate the 3D-Zernike descriptor Where n∈[0,N], l∈[0,n], m∈[-l,l]; k=(nl) / 2 and must be an integer, which is the order condition of the Nth order 3D-Zernike moment; μ∈[0,k]; Each combination of {n, l, m} that satisfies the order condition corresponds to a 3D-Zernike moment of a specific order. j is the serial number of M three-dimensional points, j = 1, 2, ..., M; is the orthogonalization coefficient; (x j ,y j ,z j ) is the coordinate of the jth point; is a harmonic function, is the arithmetic mean of the harmonic function; i is the imaginary unit; C is the combination operator; Finally, calculate all 3D-Zernike descriptors ZD nl , forming a 3DZD descriptor vector, i.e., the feature vector X E : ZD nl =||Ω nl || X E ={ZD nl } in, 8. The method for cross-source remote sensing data target recognition based on signed distance features according to claim 7, characterized in that: S4 describes the feature vector X E The process of identification through classifier is implemented based on SVM classifier.
9. The method for cross-source remote sensing data target recognition based on signed distance features according to claim 8, characterized in that: In the process of obtaining the remote sensing image to be identified as described in S1, the entire remote sensing image is first obtained, and then the target to be identified is selected from the entire remote sensing image and cut out from the entire remote sensing image as a separate target.
10. The method for cross-source remote sensing data target recognition based on signed distance features according to claim 9, characterized in that: The sdf-optical network, the sdf-infrared network and the sdf-SAR network are pre-trained, and the training processes of the sdf-optical network, the sdf-infrared network and the sdf-SAR network are the same, wherein the parameter optimization process in the sdf-optical network training includes the following steps: Calculate the loss L(θ) between the signed distance feature f6 obtained in training and the signed distance of the true value of the input target model, and optimize the parameters accordingly; L(θ)=|min(t,max(t,s)-min(t,max(t,d)| Among them, L(θ) is the required loss, θ is the learning parameter of the network model; min() is the minimum operation, max() is the maximum operation, s is the signed distance obtained by training, that is, the signed distance feature f6; d is the signed distance of the true value of the target model, and t is the set target boundary threshold.
