SAR (Synthetic Aperture Radar) data feature fusion intelligent identification method and device

Abstract

The invention discloses an SAR (Synthetic Aperture Radar) data feature fusion intelligent identification method and device, and belongs to the technical field of intelligent identification. The method comprises the following steps: preprocessing an original SAR image data set to obtain a new SAR image data set; the SAR image data set comprises a training data set and a test data set; inputting the new training data set into the multi-dimensional feature fusion network for training to obtain a target recognition model; wherein the multi-dimensional feature fusion network is constructed by a ResNet network and a traditional feature extraction network; and inputting the new test data set into the target recognition model, and outputting to obtain a target recognition result. According to the method, the sea surface ship of the SAR image can be quickly and accurately identified, and a basic method support is provided for future target multi-dimensional feature fusion identification.

Description

Technical Field

[0001] The present invention relates to the field of intelligent recognition technology, and in particular to a SAR data feature fusion intelligent recognition method and device. Background Art

[0002] Deep learning has made significant breakthroughs in fields such as speech recognition and image recognition, achieving an accuracy rate that is difficult to match with traditional recognition methods. Its application in ship SAR image target recognition can overcome the shortcomings of traditional methods such as complex calculations, long time consumption and low accuracy, and effectively improve the accuracy and efficiency of sea surface ship target recognition.

[0003] SAR images are images with richer semantic information. Existing recognition technologies can quickly extract high-level semantic features from them. Although these features can represent most of the information contained in the image, the shallow details of the image are ignored, resulting in reduced accuracy in ship recognition.

[0004] Based on this, there is an urgent need for a SAR data feature fusion intelligent recognition method and device to solve the above technical problems. Summary of the invention

[0005] The present invention provides a SAR data feature fusion intelligent recognition method and device, which can quickly and accurately identify sea surface ships in SAR images. The technical solution is as follows:

[0006] On the one hand, a SAR data feature fusion intelligent recognition method is provided, the method comprising:

[0007] Preprocessing the original SAR image data set to obtain a new SAR image data set; the SAR image data set includes a training data set and a test data set;

[0008] Inputting the new training data set into the multi-dimensional feature fusion network for training to obtain a target recognition model; wherein the multi-dimensional feature fusion network is constructed by a ResNet network and a traditional feature extraction network;

[0009] The new test data set is input into the target recognition model, and the target recognition result is output.

[0010] On the other hand, a SAR data feature fusion intelligent recognition device is provided, the device comprising:

[0011] A processing module is used to preprocess the original SAR image data set to obtain a new SAR image data set; the SAR image data set includes a training data set and a test data set;

[0012] A training module, used for inputting a new training data set into a multi-dimensional feature fusion network for training to obtain a target recognition model; wherein the multi-dimensional feature fusion network is constructed by a ResNet network and a traditional feature extraction network;

[0013] The recognition module is used to input a new test data set into the target recognition model and output a target recognition result.

[0014] On the other hand, a computer device is provided, which includes a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the steps of the above-mentioned SAR data feature fusion intelligent recognition method.

[0015] On the other hand, a computer-readable storage medium is provided, wherein a computer program is stored in the storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned SAR data feature fusion intelligent recognition method are implemented.

[0016] On the other hand, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned SAR data feature fusion intelligent recognition method.

[0017] The embodiment of the present invention provides a SAR data feature fusion intelligent recognition method and device, firstly obtains a preprocessed training data set, inputs the training data set into a multidimensional feature fusion recognition network constructed by a ResNet backbone network and a traditional feature extraction network for training, obtains a trained target recognition model, and then inputs the preprocessed test data set into the target recognition model to obtain a recognition result. This method combines traditional features with the multidimensional feature fusion recognition method of deep network features itself, can quickly and accurately identify SAR image sea surface ships, and provides a basic method support for future target multidimensional feature fusion recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is a flow chart of a SAR data feature fusion intelligent recognition method provided by an embodiment of the present invention;

[0020] Figure 2 is a schematic diagram of a new SAR image data set provided by an embodiment of the present invention;

[0021] Figure 3 It is a schematic diagram of a multi-dimensional feature fusion network structure provided by an embodiment of the present invention;

[0022] Figure 4 It is a schematic diagram of the recognition result of the model provided by one embodiment of the present invention for recognizing a surface cargo ship target;

[0023] Figure 5 It is a structural diagram of a SAR data feature fusion intelligent recognition device provided by an embodiment of the present invention;

[0024] Figure 6 It is a hardware architecture diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0026] As mentioned above, existing technologies often only use a single image feature to identify the target, which causes some information representing image details to be ignored during the recognition process, resulting in low target recognition accuracy.

[0027] Based on this, the concept of the present invention is to utilize the fast learning ability of the deep learning network and integrate traditional features into the network to enhance its learning ability, thereby realizing fast and accurate recognition of SAR images of surface ships.

[0028] The specific implementation of the above concept is described below.

[0029] Please refer to Figure 1 , an embodiment of the present invention provides a SAR data feature fusion intelligent recognition method, the method comprising:

[0030] Step 100, preprocessing the original SAR image data set to obtain a new SAR image data set; the SAR image data set includes a training data set and a test data set;

[0031] Step 102, inputting the new training data set into a multi-dimensional feature fusion network for training to obtain a target recognition model; wherein the multi-dimensional feature fusion network is constructed by a ResNet network and a traditional feature extraction network;

[0032] Step 104: input the new test data set into the target recognition model, and output the target recognition result.

[0033] In the embodiment of the present invention, a pre-processed training data set is first obtained, and the training data set is input into a multi-dimensional feature fusion recognition network constructed by a ResNet backbone network and a traditional feature extraction network for training to obtain a trained target recognition model, and then the pre-processed test data set is input into the target recognition model to obtain a recognition result. This method combines traditional features with the multi-dimensional feature fusion recognition method of the deep network's own features, and can quickly and accurately identify surface ships in SAR images, providing a basic method support for future target multi-dimensional feature fusion recognition.

[0034] Described below Figure 1 How the various steps are performed.

[0035] First, with respect to step 100 , the original SAR image dataset is preprocessed to obtain a new SAR image dataset.

[0036] SAR images are different from natural images and have richer semantic information. The SAR data of ships are high-resolution images that contain rich detail information. In order to ensure the accuracy of subsequent target recognition, three types of ship images with different azimuths and a pitch angle of 15° and minimal noise are selected as the original SAR image dataset.

[0037] Furthermore, considering the data imbalance problem of the original image dataset, the original data is augmented to enhance the data and expand the samples. Data augmentation can not only improve the generalization ability of the model to prevent overfitting, but also improve the robustness of the model. It is especially suitable for target classification and recognition tasks because its category information does not change with the transformation of the image.

[0038] In an embodiment of the present invention, the data augmentation processing includes symmetric processing of the image about the horizontal mirror plane, rotating the image by 90° and symmetric processing of the image about the vertical mirror plane. Three different samples are obtained through these steps, which together constitute a sample image data set.

[0039] Furthermore, we add corresponding labels to each image data in the sample image dataset, and obtain Figure 2 As shown, a new dataset containing sample images and data labels.

[0040] It is worth noting that in order to facilitate the subsequent training of the model, the data set needs to be divided into a training data set and a test data set, that is, the training data set is used to train the subsequent model, and the test data set is used to detect the accuracy of the model. Both data sets adopt the above data processing.

[0041] Then, for step 102, the new training data set is input into the multi-dimensional feature fusion network for training to obtain a target recognition model.

[0042] In the feature analysis of ship targets in SAR images, electromagnetic scattering feature extraction is one of the key steps. Electromagnetic scattering features are based on the physical principles of SAR imaging and reflect the interaction between target objects and electromagnetic waves. These features include the strong scattering center, angular reflection, surface roughness, etc. of the target. In practical applications, these features can help distinguish different types of targets, such as different types of ships.

[0043] Taking into account that the deep feature map has a large scale and can capture the high-level semantic information of the salient object; and the shallow feature map has a small scale and can extract the fine semantic information of the target, therefore, in the embodiment of the present invention, the ResNet network is used as the backbone network to establish the first network branch; the traditional feature extraction algorithm is used as the core algorithm to establish the second network branch; and the output end of the first network branch is connected to the output end of the second network branch to obtain a multi-dimensional feature fusion network.

[0044] like Figure 3 As shown, Figure 3 It is a schematic diagram of the multi-dimensional feature fusion network structure provided in this embodiment. The first layer of the ResNet network is a convolution layer with a convolution kernel of 7×7. In order to extract features useful for recognition, a CBAM attention module is added after the convolution layer, and the dimension of its output feature map is the same as the input dimension.

[0045] Then connect to the ResNet network, where Layer 1 consists of 3 Building Blocks, Layer 2 consists of 4 Building Blocks, Layer 3 consists of 6 Building Blocks, and Layer 4 consists of 4 Building Blocks. The ResNet network increases the depth of the network by stacking Building Blocks, giving the network stronger expressive power.

[0046] The building block is composed of two 3×3 convolutional layers stacked together, and each convolutional layer is followed by a batch normalization layer and a ReLU function. The role of batch normalization is to unify the distribution of each batch of data so that the distribution of input data can fall into the sensitive area of ​​the nonlinear activation function to accelerate network convergence and improve stability. For the ReLU function, batch normalization can more effectively prevent gradient explosion and gradient disappearance.

[0047] For the second network branch, the traditional Harris corner detection algorithm is used for feature extraction. The algorithm is based on the gradient change of the image and detects feature points (such as corner points) by calculating the autocorrelation matrix of the local window in the image.

[0048] Finally, by connecting the output ends of the two network branches together, a multi-dimensional feature fusion network for target recognition can be constructed.

[0049] After obtaining the multidimensional feature fusion network, the new training data set established by the above process can be used to train it. The training process includes: using the category balance loss algorithm to update the weights of the multidimensional feature fusion network to solve the training problem of data category imbalance; using the stochastic gradient descent algorithm to train the multidimensional feature fusion network to obtain training results; wherein, the training learning rate is set to 0.005, the weight decay is set to 0.0005, and the momentum is set to 0.9; judging whether the training results meet the convergence conditions, if so, ending the training to obtain the target recognition model.

[0050] Specifically, the network introduces Class Balance Loss (CB Loss) to overcome the class imbalance problem. CB Loss introduces a weighting factor that is inversely proportional to the number of "valid samples" in each class to reallocate the weights of the losses for each class, thereby obtaining a class-balanced loss and solving the problem of training unbalanced data.

[0051] Furthermore, the network uses the Stochastic Gradient Descent (SGD) algorithm to train the model, with the learning rate set to 0.005, the weight decay set to 0.0005, and the momentum set to 0.9. The training is performed until the convergence conditions are met to obtain a target recognition model. The specific training process is well known to those skilled in the art and will not be described in detail here.

[0052] With respect to step 104, the new test data set is input into the target recognition model, and the target recognition result is output.

[0053] After obtaining the target recognition model, a new test data set is input into it, and the first network branch is used to extract features of the new test data set to obtain a high-dimensional feature vector of the target to be identified; the second network branch is used to extract features of the new test data set to obtain the Harris corner feature vector of the target to be identified; the high-dimensional feature vector and the Harris corner feature vector are concatenated end to end to obtain a multi-dimensional feature fusion vector; the multi-dimensional feature fusion vector is normalized and output to obtain the target recognition result.

[0054] like Figure 4 As shown, Figure 4The model provided in this embodiment identifies the recognition result of the surface cargo ship target. The recognition rate of the above new test data set after the target recognition model is better than 80%, which proves that the above method can quickly and accurately identify the surface ships in the SAR image, and provide a basic method support for the future target multi-dimensional feature fusion recognition.

[0055] Please refer to Figure 5 The embodiment of the present invention provides a SAR data feature fusion intelligent recognition device, which includes:

[0056] The processing module 500 is used to preprocess the original SAR image data set to obtain a new SAR image data set; the SAR image data set includes a training data set and a test data set;

[0057] The training module 502 is used to input the new training data set into the multi-dimensional feature fusion network for training to obtain a target recognition model; wherein the multi-dimensional feature fusion network is constructed by a ResNet network and a traditional feature extraction network;

[0058] The recognition module 504 is used to input the new test data set into the target recognition model and output the target recognition result.

[0059] In the embodiment of the present invention, when the processing module 500 performs preprocessing on the original SAR image data set to obtain a new SAR image data set, it is specifically used to perform the following operations: select three types of ship images with different azimuths and minimum sound with a pitch angle of 15° as the original SAR image data set; perform augmentation processing on the original SAR image data set to obtain a sample image data set; add a corresponding label to each image data in the sample image data set to obtain a new data set including the sample image and the data label.

[0060] In the embodiment of the present invention, the augmentation process includes performing symmetric processing on the image about the horizontal mirror plane, rotating the image by 90 degrees, and performing symmetric processing on the image about the vertical mirror plane.

[0061] In an embodiment of the present invention, a multidimensional feature fusion network is constructed in the following manner: a first network branch is established using a ResNet network as a backbone network; a second network branch is established using a traditional feature extraction algorithm as a core algorithm; and an output end of the first network branch and an output end of the second network branch are connected to obtain the multidimensional feature fusion network.

[0062] In an embodiment of the present invention, when the training module 502 inputs a new training data set into the multidimensional feature fusion network for training to obtain a target recognition model, it is specifically used to perform the following operations: using a category balance loss algorithm to update the weights of the multidimensional feature fusion network to solve the training problem of data category imbalance; using a stochastic gradient descent algorithm to train the multidimensional feature fusion network to obtain a training result; wherein the training learning rate is set to 0.005, the weight attenuation is set to 0.0005, and the momentum is set to 0.9; judging whether the training result meets the convergence condition, and if so, terminating the training to obtain the target recognition model.

[0063] In the embodiment of the present invention, when the recognition module 504 inputs the new test data set into the target recognition model and outputs the target recognition result, it is specifically used to perform the following operations: using the first network branch to extract features of the new test data set to obtain a high-dimensional feature vector of the target to be identified; using the second network branch to extract features of the new test data set to obtain a Harris corner feature vector of the target to be identified; concatenating the high-dimensional feature vector and the Harris corner feature vector end to end to obtain a multi-dimensional feature fusion vector; normalizing the multi-dimensional feature fusion vector and outputting the target recognition result.

[0064] It should be noted that: the SAR data feature fusion intelligent recognition device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the SAR data feature fusion intelligent recognition device and the SAR data feature fusion intelligent recognition method embodiment provided in the above embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0065] The embodiment of the present application also provides a computer device, please refer to Figure 6 The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the SAR data feature fusion intelligent recognition method provided by the above-mentioned method embodiments.

[0066] An embodiment of the present application also provides a computer-readable storage medium, on which is stored at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the SAR data feature fusion intelligent recognition method provided by the above-mentioned method embodiments.

[0067] An embodiment of the present application also provides a computer program product, which includes a computer program. A processor of a computer device reads the computer program from a computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the SAR data feature fusion intelligent recognition method described in any of the above embodiments.

[0068] For the convenience of description, the above system or device is described by dividing it into various modules or units according to its functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0069] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0070] Finally, it should be noted that, in this article, relational terms such as first, second, third and fourth are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0071] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A SAR data feature fusion intelligent recognition method, characterized in that: include: Preprocessing the original SAR image data set to obtain a new SAR image data set; The SAR image data set includes a training data set and a test data set; Inputting the new training data set into the multi-dimensional feature fusion network for training to obtain a target recognition model; wherein the multi-dimensional feature fusion network is constructed by a ResNet network and a traditional feature extraction network; The new test data set is input into the target recognition model, and the target recognition result is output.

2. The method according to claim 1, characterized in that The preprocessing of the original SAR image data set to obtain a new SAR image data set includes: The three types of ship images with different azimuth angles and the least noise at a pitch angle of 15° were selected as the original SAR image dataset; Perform augmentation processing on the original SAR image data set to obtain a sample image data set; Add a corresponding label to each image data in the sample image dataset to obtain a new dataset containing sample images and data labels.

3. The method according to claim 2, characterized in that The augmentation process includes performing symmetric processing on the image with respect to a horizontal mirror plane, performing a 90° rotation process on the image, and performing symmetric processing on the image with respect to a vertical mirror plane.

4. The method according to claim 1, characterized in that The multi-dimensional feature fusion network is constructed in the following way: Use the ResNet network as the backbone network to establish the first network branch; Using the traditional feature extraction algorithm as the core algorithm, the second network branch is established; The output end of the first network branch and the output end of the second network branch are connected to obtain the multi-dimensional feature fusion network.

5. The method according to claim 4, characterized in that The new training data set is input into the pre-built multi-dimensional feature fusion network for training to obtain the target recognition model, including: The class balance loss algorithm is used to update the weights of the multi-dimensional feature fusion network to solve the training problem of data class imbalance; The multi-dimensional feature fusion network was trained using the stochastic gradient descent algorithm to obtain the training results; the training learning rate was set to 0.005, the weight decay was set to 0.0005, and the momentum was set to 0.9; It is determined whether the training result meets the convergence condition, and if so, the training is terminated to obtain the target recognition model.

6. The method according to claim 5, characterized in that The step of inputting a new test data set into the target recognition model and outputting a target recognition result comprises: Using the first network branch to extract features from a new test data set to obtain a high-dimensional feature vector of the target to be identified; Using the second network branch to extract features from a new test data set, and obtaining a Harris corner feature vector of the target to be identified; Concatenate the high-dimensional feature vector and the Harris corner feature vector end to end to obtain a multi-dimensional feature fusion vector; The multi-dimensional feature fusion vector is normalized and output to obtain a target recognition result.

7. A SAR data feature fusion intelligent recognition device, characterized in that: include: A processing module is used to preprocess the original SAR image data set to obtain a new SAR image data set; the SAR image data set includes a training data set and a test data set; A training module, used for inputting a new training data set into a multi-dimensional feature fusion network for training to obtain a target recognition model; wherein the multi-dimensional feature fusion network is constructed by a ResNet network and a traditional feature extraction network; The recognition module is used to input a new test data set into the target recognition model and output a target recognition result.

8. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed in a computer, the computer is caused to execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises a computer program, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images