A method and device for target detection in synthetic aperture radar images

Abstract

This application discloses a method and device for target detection in synthetic aperture radar images. The method includes determining at least two convolution operations and constructing a first neural network based on the at least two convolution operations; inputting a sample synthetic aperture radar image into the first neural network and calculating a first loss value based on the predicted result and the actual sample result; training based on the first loss value and deriving a second neural network based on the architecture parameters and preset derivation rules in the first neural network; inputting the unprocessed synthetic aperture radar image into the second neural network and displaying a target detection frame on the unprocessed synthetic aperture radar image based on the predicted result. Compared to manually designed neural network models for target detection, neural architecture search automates the design of networks that better meet the requirements, effectively avoiding the domain knowledge limitations caused by excessive expert manual knowledge and significantly improving the efficiency and accuracy of target detection.

Description

Technical Field

[0001] The present application belongs to the technical field of ship target detection, and in particular relates to a target detection method and device for synthetic aperture radar images. Background Art

[0002] In the fields of machine learning and cognitive science, a neural network is a mathematical or computational model that mimics the structure and function of biological neural networks and is used to estimate or approximate complex functions. Deep learning, based on deep neural networks, aims to enhance function fitting capabilities using multi-layer neural networks. Several types of deep neural networks, such as convolutional neural networks and recurrent neural networks, have been developed. Deep neural networks have achieved remarkable results in the field of object detection. Deep learning algorithms are divided into two processes: training and inference. The training process, based on large amounts of data, continuously adjusts the network's parameters through forward computation (including matrix multiplications, convolutions, and recurrent layers) and backward updates (primarily gradient operations) to improve the model's performance on specific tasks. The inference process, on the other hand, keeps the neural network parameters unchanged and primarily involves forward computation. Because deep convolutional neural networks can learn key features from raw data, they have achieved significant success in fields such as object detection and are increasingly being applied to object detection in synthetic aperture radar images.

[0003] Currently, many research works that apply deep neural networks to synthetic aperture radar images simply apply deep neural network models from other works to synthetic aperture radar images of ship targets. They do not specifically design and adjust the deep neural network models based on the synthetic aperture radar image target detection task of ship targets. As a result, the migrated deep neural network models cannot adapt well to the new ship target detection task, which in turn affects the overall computational efficiency and performance. Summary of the Invention

[0004] This application aims to solve the above-mentioned problem that many current research works on applying deep neural networks to synthetic aperture radar images simply apply deep neural network models from other works to synthetic aperture radar images of ship targets, without specifically designing and adjusting the deep neural network models according to the synthetic aperture radar image target detection task of ship targets. As a result, the migrated deep neural network models cannot adapt well to the new ship target detection task, thereby affecting the overall computational efficiency and performance. A target detection method and device for synthetic aperture radar images are proposed, including:

[0005] Determining at least two convolution operations, and constructing a first neural network based on the at least two convolution operations;

[0006] Inputting the sample synthetic aperture radar image into the first neural network, and calculating a first loss value based on the prediction result of the first neural network and the actual result of the sample;

[0007] Training the first neural network based on the first loss value, and deriving the second neural network according to the architecture parameters of the trained first neural network and a preset derivation rule;

[0008] The synthetic aperture radar image to be processed is input into the second neural network, and a target detection frame is displayed on the synthetic aperture radar image to be processed according to the prediction result of the second neural network.

[0009] In an optional solution of the first aspect, the first neural network includes two stem layers and eight unit layers, wherein:

[0010] The nth unit layer includes a first preprocessing node, a second preprocessing node, and a first intermediate node, a second intermediate node, a third intermediate node, and a fourth intermediate node arranged in sequence, the input end of the first preprocessing node is connected to the output end of the (n-2)th unit layer, and the output end of the first preprocessing node is connected to the input end of the first intermediate node, the input end of the second intermediate node, the input end of the third intermediate node, and the input end of the fourth intermediate node, respectively;

[0011] The input end of the second preprocessing node is connected to the output end of the (n-1)th unit layer, and the output end of the second preprocessing node is connected to the input end of the first intermediate node, the input end of the second intermediate node, the input end of the third intermediate node, and the input end of the fourth intermediate node respectively;

[0012] The output terminal of the first intermediate node is connected to the input terminal of the second intermediate node, the input terminal of the third intermediate node, and the input terminal of the fourth intermediate node respectively;

[0013] The output terminal of the second intermediate node is connected to the input terminal of the third intermediate node and the input terminal of the fourth intermediate node respectively;

[0014] An output terminal of the third intermediate node is connected to an input terminal of the fourth intermediate node;

[0015] The output end of the nth unit layer is used to splice the output results of the first intermediate node, the output results of the second intermediate node, the output results of the third intermediate node, and the output results of the fourth intermediate node, and output the processed output results to the second preprocessing node of the n+1th unit layer, where n is a positive integer greater than or equal to 3.

[0016] In another optional solution of the first aspect, the two stem layers include a first stem layer and a second stem layer, wherein:

[0017] The input end of the first preprocessing node in the first unit layer is connected to the output end of the first stem layer, and the input end of the second preprocessing node in the first unit layer is connected to the output end of the second stem layer;

[0018] The input end of the first preprocessing node in the second unit layer is connected to the output end of the second stem layer, and the input end of the second preprocessing node in the second unit layer is connected to the output end of the first unit layer.

[0019] In another optional solution of the first aspect, training the first neural network based on the first loss value includes:

[0020] Fixing weight parameters of the first neural network and updating architecture parameters of the first neural network based on the first loss value;

[0021] After the architecture parameters of the first neural network are updated, the sample synthetic aperture radar image is re-input into the updated first neural network, and a second loss value is calculated based on the prediction result of the updated first neural network and the true result of the sample;

[0022] The architecture parameters of the updated first neural network are fixed, and the updated first neural network is updated again based on the second loss value.

[0023] In another optional solution of the first aspect, before inputting the synthetic aperture radar image to be processed into the second neural network, the method further includes:

[0024] Inputting the sample synthetic aperture radar image into the second neural network, and calculating a third loss value based on the prediction result of the second neural network and the true result of the sample;

[0025] training the second neural network based on the third loss value;

[0026] Inputting the synthetic aperture radar image to be processed into the second neural network includes:

[0027] The synthetic aperture radar image to be processed is input into the trained second neural network.

[0028] In another optional solution of the first aspect, the prediction results of the second neural network include the horizontal and vertical coordinates, offsets, and confidence levels of the center points of all target detection boxes in the synthetic aperture radar image to be processed;

[0029] Based on the prediction results of the second neural network, a target detection frame is displayed on the synthetic aperture radar image to be processed, including:

[0030] Filter the target detection frames whose confidence is lower than the preset confidence threshold;

[0031] Perform non-maximum suppression on all filtered target detection frames;

[0032] All target detection frames that have undergone non-maximum suppression processing, the horizontal and vertical coordinates of the center point corresponding to each target detection frame, the offset, and the confidence level are displayed on the synthetic aperture radar image to be processed.

[0033] In yet another alternative of the first aspect, the second neural network includes two stem layers and fourteen unit layers.

[0034] In a second aspect, an embodiment of the present application provides a target detection device for synthetic aperture radar images, comprising:

[0035] a first network construction module, configured to determine at least two convolution operations and construct a first neural network based on the at least two convolution operations;

[0036] a loss calculation module, configured to input the sample synthetic aperture radar image into the first neural network and calculate a first loss value based on the prediction result of the first neural network and the actual result of the sample;

[0037] A second network construction module is used to train the first neural network based on the first loss value, and derive a second neural network according to the architecture parameters of the trained first neural network and a preset derivation rule;

[0038] The result display module is used to input the synthetic aperture radar image to be processed into the second neural network and display the target detection frame on the synthetic aperture radar image to be processed according to the prediction result of the second neural network.

[0039] In an optional solution of the second aspect, the first neural network includes two stem layers and eight unit layers, wherein:

[0040] The nth unit layer includes a first preprocessing node, a second preprocessing node, and a first intermediate node, a second intermediate node, a third intermediate node, and a fourth intermediate node arranged in sequence, the input end of the first preprocessing node is connected to the output end of the (n-2)th unit layer, and the output end of the first preprocessing node is connected to the input end of the first intermediate node, the input end of the second intermediate node, the input end of the third intermediate node, and the input end of the fourth intermediate node, respectively;

[0041] The input end of the second preprocessing node is connected to the output end of the (n-1)th unit layer, and the output end of the second preprocessing node is connected to the input end of the first intermediate node, the input end of the second intermediate node, the input end of the third intermediate node, and the input end of the fourth intermediate node respectively;

[0042] The output terminal of the first intermediate node is connected to the input terminal of the second intermediate node, the input terminal of the third intermediate node, and the input terminal of the fourth intermediate node respectively;

[0043] The output terminal of the second intermediate node is connected to the input terminal of the third intermediate node and the input terminal of the fourth intermediate node respectively;

[0044] An output terminal of the third intermediate node is connected to an input terminal of the fourth intermediate node;

[0045] The output end of the nth unit layer is used to splice the output results of the first intermediate node, the output results of the second intermediate node, the output results of the third intermediate node, and the output results of the fourth intermediate node, and output the processed output results to the second preprocessing node of the n+1th unit layer, where n is a positive integer greater than or equal to 3.

[0046] In another optional solution of the second aspect, the two stem layers include a first stem layer and a second stem layer, wherein:

[0047] The input end of the first preprocessing node in the first unit layer is connected to the output end of the first stem layer, and the input end of the second preprocessing node in the first unit layer is connected to the output end of the second stem layer;

[0048] The input end of the first preprocessing node in the second unit layer is connected to the output end of the second stem layer, and the input end of the second preprocessing node in the second unit layer is connected to the output end of the first unit layer.

[0049] In another optional solution of the second aspect, the second network construction module includes:

[0050] Fixing weight parameters of the first neural network and updating architecture parameters of the first neural network based on the first loss value;

[0051] After the architecture parameters of the first neural network are updated, the sample synthetic aperture radar image is re-input into the updated first neural network, and a second loss value is calculated based on the prediction result of the updated first neural network and the true result of the sample;

[0052] The architecture parameters of the updated first neural network are fixed, and the updated first neural network is updated again based on the second loss value.

[0053] In yet another optional solution of the second aspect, the apparatus further comprises:

[0054] Before inputting the to-be-processed synthetic aperture radar image into the second neural network, inputting the sample synthetic aperture radar image into the second neural network, and calculating a third loss value based on a prediction result of the second neural network and a true result of the sample;

[0055] training the second neural network based on the third loss value;

[0056] Inputting the synthetic aperture radar image to be processed into the second neural network includes:

[0057] The synthetic aperture radar image to be processed is input into the trained second neural network.

[0058] In another optional solution of the second aspect, the prediction results of the second neural network include the horizontal and vertical coordinates, offsets, and confidence levels of the center points of all target detection boxes in the synthetic aperture radar image to be processed;

[0059] The result display module includes:

[0060] Filter the target detection frames whose confidence is lower than the preset confidence threshold;

[0061] Perform non-maximum suppression on all filtered target detection frames;

[0062] All target detection frames that have undergone non-maximum suppression processing, the horizontal and vertical coordinates of the center point corresponding to each target detection frame, the offset, and the confidence level are displayed on the synthetic aperture radar image to be processed.

[0063] In yet another optional embodiment of the second aspect, the second neural network includes two stem layers and fourteen unit layers.

[0064] In a third aspect, an embodiment of the present application further provides a target detection device for synthetic aperture radar images, comprising a processor and a memory;

[0065] The processor is connected to the memory;

[0066] a memory for storing executable program code;

[0067] The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the target detection method for synthetic aperture radar images provided by the first aspect of the embodiment of the present application or any one of the implementation methods of the first aspect.

[0068] In a fourth aspect, an embodiment of the present application provides a computer storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, the method for target detection for synthetic aperture radar images provided by the first aspect of the embodiment of the present application or any one of the implementation methods of the first aspect can be implemented.

[0069] In an embodiment of the present application, when detecting a ship target, at least two convolution operations can be determined, and a first neural network can be constructed based on the at least two convolution operations; a sample synthetic aperture radar image can be input into the first neural network, and a first loss value can be calculated based on the prediction results of the first neural network and the actual sample results; the first neural network can be trained based on the first loss value, and a second neural network can be derived based on the architecture parameters of the trained first neural network and preset derivation rules; the synthetic aperture radar image to be processed can be input into the second neural network, and a target detection box can be displayed on the synthetic aperture radar image to be processed based on the prediction results of the second neural network. Compared with manually designed neural network models for target detection, the neural architecture search can automatically complete the network design that better meets the requirements, effectively avoiding the domain knowledge limitations caused by excessive expert manual knowledge and greatly improving the efficiency and accuracy of target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0071] Figure 1 A schematic diagram of the overall process of a target detection method for synthetic aperture radar images provided in an embodiment of the present application;

[0072] Figure 2 A schematic structural diagram of a unit layer provided in an embodiment of the present application;

[0073] Figure 3 A schematic diagram of the structure of a neural network provided in an embodiment of the present application;

[0074] Figure 4 A schematic diagram of target detection in a synthetic aperture radar image provided in an embodiment of the present application;

[0075] Figure 5 A schematic diagram of the structure of a target detection device for synthetic aperture radar images provided in an embodiment of the present application;

[0076] Figure 6 A schematic structural diagram of another target detection device for synthetic aperture radar images provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0078] In the following introduction, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. The following introduction provides multiple embodiments of the present application. Different embodiments can be replaced or combined, so the present application can also be considered to include all possible combinations of the same and / or different embodiments described. Therefore, if one embodiment includes features A, B, and C, and another embodiment includes features B and D, then the present application should also be considered to include embodiments containing one or more of all other possible combinations of A, B, C, and D, even though the embodiment may not be clearly described in the following text.

[0079] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made to the function and arrangement of the elements described without departing from the scope of the present application. Various examples may appropriately omit, replace, or add various processes or components. For example, the described method may be performed in an order different from the order described, and various steps may be added, omitted, or combined. In addition, features described in some examples may be combined in other examples.

[0080] See also Figure 1 , Figure 1 The figure shows an overall flow chart of a target detection method for synthetic aperture radar images provided by an embodiment of the present application.

[0081] like Figure 1 As shown, the target detection method for synthetic aperture radar images may include at least the following steps:

[0082] Step 102: Determine at least two convolution operations, and construct a first neural network based on the at least two convolution operations.

[0083] In an embodiment of the present application, the target detection method for synthetic aperture radar images can be used to process the acquired synthetic aperture radar images and display the detection results of ship targets on the synthetic aperture radar images, wherein the detection results of the ship targets may include but are not limited to including a target detection box for selecting a ship, the horizontal and vertical coordinates of the center point of the target detection box, the offset and the confidence level.

[0084] Specifically, when detecting ship targets in synthetic aperture radar images, at least two convolution operations for constructing a neural network can be determined first, wherein the at least two convolution operations can correspond to commonly used and advanced convolution operations in the field of deep neural networks, which can include but are not limited to any at least two of the depthwise separable convolution (Sep_conv_3) operation with a convolution kernel size of 3, the depthwise separable convolution (Sep_conv_5) operation with a convolution kernel size of 5, the dilated convolution (Dil_conv_3) operation with a convolution kernel size of 3, the dilated convolution (Dil_conv_5) operation with a convolution kernel size of 5, average pooling (Avg_pool), maximum pooling (Max_pool), skip_connection and none operations, and in the embodiment of the present application, it is preferred that the above-mentioned eight convolution operations can all be used as at least two convolution operations for constructing a neural network.

[0085] Furthermore, after determining at least two convolution operations for constructing a neural network, a first neural network can be constructed based on the at least two convolution operations based on the characteristics of neural architecture search (also known as NAS). NAS can be understood as modeling architecture search as a nested two-layer optimization problem, where the inner loop optimizes the network weights on a training set given a given network architecture, while the outer loop fixes the network weights and optimizes the representation of the network architecture on a validation set. This will not be described in detail in the embodiments of this application.

[0086] It can be understood that the first neural network may include at least two stem layers and eight unit layers. For the nth unit layer, the nth unit layer includes a first preprocessing node, a second preprocessing node, and a first intermediate node, a second intermediate node, a third intermediate node, and a fourth intermediate node that are sequentially arranged. The input end of the first preprocessing node is connected to the output end of the n-2th unit layer, and the output end of the first preprocessing node is connected to the input end of the first intermediate node, the input end of the second intermediate node, the input end of the third intermediate node, and the input end of the fourth intermediate node, respectively.

[0087] The input end of the second preprocessing node is connected to the output end of the (n-1)th unit layer, and the output end of the second preprocessing node is connected to the input end of the first intermediate node, the input end of the second intermediate node, the input end of the third intermediate node, and the input end of the fourth intermediate node respectively;

[0088] The output terminal of the first intermediate node is connected to the input terminal of the second intermediate node, the input terminal of the third intermediate node, and the input terminal of the fourth intermediate node respectively;

[0089] The output terminal of the second intermediate node is connected to the input terminal of the third intermediate node and the input terminal of the fourth intermediate node respectively;

[0090] An output terminal of the third intermediate node is connected to an input terminal of the fourth intermediate node;

[0091] The output end of the nth unit layer is used to splice the output results of the first intermediate node, the output results of the second intermediate node, the output results of the third intermediate node and the output results of the fourth intermediate node, and output the processed output results to the second preprocessing node of the n+1th unit layer.

[0092] See here Figure 2 The structural diagram of a unit layer provided by an embodiment of the present application is shown. Figure 2 As shown, the nth unit layer may include preprocessing node 0, preprocessing node 1, and intermediate nodes 2, intermediate nodes 3, intermediate nodes 4, and intermediate nodes 5, which are arranged in sequence. Preprocessing node 0 may process the output results of the n-2th unit layer and output the processed results to intermediate nodes 2, intermediate nodes 3, intermediate nodes 4, and intermediate nodes 5, respectively. Preprocessing node 1 may process the output results of the n-1th unit layer and output the processed results to intermediate nodes 2, intermediate nodes 3, intermediate nodes 4, and intermediate nodes 5, respectively. Among them, intermediate node 3 may also obtain the intermediate results output by intermediate node 2, intermediate node 4 may also obtain the intermediate results output by intermediate node 2 and the intermediate results output by intermediate node 3, and intermediate node 5 may also obtain the intermediate results output by intermediate node 2, the intermediate results output by intermediate node 3, and the intermediate results output by intermediate node 4.

[0093] It can be understood that, taking the example of intermediate node 2 outputting the intermediate result to intermediate node 3, the connecting edge between intermediate node 2 and intermediate node 3 can correspond to the eight convolution operations mentioned above, and each convolution operation is assigned a weight value (the sum of the weight values of the eight convolution operations is 1). The intermediate result output by intermediate node 2 first needs to be calculated by all eight convolution operations respectively, and then the calculation results of the eight convolution operations are weighted and summed, and the summed result, the processing result input by preprocessing node 1, and the processing result input by preprocessing node 2 are weighted and summed again to obtain the final intermediate result for output to intermediate node 3. It should be noted that, of all the input ends of any intermediate node here (that is, corresponding to the input edges), only two of the input ends need to be retained according to the preset search target setting. Therefore, when the first neural network is initialized, all the input ends of each intermediate node will also be assigned a weight, and the sum of the input end weights of each intermediate node is 1.

[0094] After obtaining the intermediate results of intermediate node 2, the intermediate results of intermediate node 3, the intermediate results of intermediate node 4 and the intermediate results of intermediate node 5 respectively, the intermediate results of intermediate node 2, the intermediate results of intermediate node 3, the intermediate results of intermediate node 4 and the intermediate results of intermediate node 5 can be spliced to obtain the final output result of the nth unit layer, and the final output result can be, but is not limited to, output to the second preprocessing node of the n+1th unit layer, and then the final output result of the n+1th unit layer is calculated.

[0095] It should be noted that the n mentioned above is a positive integer greater than or equal to 3, which means that starting from the third unit layer, the final output result can be calculated by referring to the above-mentioned method. For the first unit layer, the input end of the first preprocessing node in the first unit layer can be connected to the output end of the first stem layer in the first neural network to obtain the processing result of the first stem layer in the first neural network; the input end of the second preprocessing node in the first unit layer can be connected to the output end of the second stem layer in the first neural network to obtain the processing result of the second stem layer in the first neural network, and the structure of the first unit layer and the output result calculation method can be consistent with the structure and output result calculation method of the above-mentioned nth unit layer.

[0096] For the second unit layer, the input end of the first preprocessing node in the second unit layer can be connected to the output end of the second stem layer in the first neural network to obtain the processing result of the second stem layer in the first neural network; the input end of the second preprocessing node in the first unit layer can be connected to the output end of the first unit layer to obtain the output result of the first unit layer, and the structure of the second unit layer and the output result calculation method can be consistent with the structure and output result calculation method of the above-mentioned nth unit layer.

[0097] Also see here Figure 3 The schematic diagram of the structure of a neural network provided by the embodiment of the present application is shown in FIG. Figure 3 As shown in the upper part of , the first neural network can specifically include an input unit, a first stem layer, a second stem layer, a first normal unit layer, a second normal unit layer, a first reduced unit layer, a third normal unit layer, a fourth normal unit layer, a second reduced unit layer, a fifth normal unit layer, a sixth normal unit layer, and a detection head unit composed of ordinary convolution with a relu activation function. Among them, the size and number of channels of the image input to the normal unit layer will not change, which can be understood as the feature dimension has not changed; the size of the image input to the reduced unit layer is reduced by half, and the number of channels is doubled.

[0098] Step 104: Input the sample synthetic aperture radar image into the first neural network, and calculate a first loss value based on the prediction result of the first neural network and the actual result of the sample.

[0099] Specifically, after constructing the first neural network, to ensure the prediction accuracy of the first neural network, a sample synthetic aperture radar image can be input into the first neural network unit to obtain the prediction result of the first neural network for the sample synthetic aperture radar image. This prediction result is then combined with the sample actual result corresponding to the sample synthetic aperture radar image to calculate a first loss value using a predetermined loss function. It is understood that in the process of calculating the first loss value, the sum of the weighted theoretical computational effort of the first neural network can be calculated based on, but is not limited to, the weight distribution of each edge in the first neural network and the weight distribution of each edge between intermediate nodes. The logarithm of the theoretical computational effort of the first neural network is then taken to ensure that the sum and the loss value calculated based on the prediction result and the sample actual result are of the same order of magnitude. Finally, the loss value calculated based on the prediction result and the sample actual result and the weighted theoretical computational effort of the first neural network are added according to the predetermined weight term to obtain the final first loss value.

[0100] The weighted theoretical calculation amount of the first neural network is added here to make the optimization direction of the first neural network appropriately biased towards operations with smaller calculation amount. The loss value calculated based on the prediction results and the actual results of the samples can ensure that the optimization direction of the first neural network does not ignore the target detection ability of the network.

[0101] Step 106: Train the first neural network based on the first loss value, and derive the second neural network according to the architecture parameters of the trained first neural network and the preset derivation rules.

[0102] Specifically, after calculating the first loss value of the first neural network, the first neural network can be iteratively trained based on the first loss value, and a second neural network corresponding to the first neural network can be obtained based on the architectural parameters of the trained first neural network and a preset derivation rule. The preset derivation rule can be understood as: multiplying the weight of each edge in the first neural network by the weight of the operation on the edge, and then, based on this new weight distribution, selecting the two edges with the largest new weights for each intermediate node's input edge (the operations on the edges also select the ones with the largest weights).

[0103] In the embodiment of the present application, the second neural network may include at least two stem layers and fourteen unit layers. Figure 3 The schematic diagram of the structure of a neural network provided by the embodiment of the present application is shown in FIG. Figure 3As shown in the lower part of , the second neural network can specifically include an input unit, a first stem layer, a second stem layer, a first normal unit layer, a second normal unit layer, a third normal unit layer, a fourth normal unit layer, a first reduced unit layer, a fifth normal unit layer, a sixth normal unit layer, a seventh normal unit layer, an eighth normal unit layer, a second reduced unit layer, a ninth normal unit layer, a tenth normal unit layer, an eleventh normal unit layer, a twelfth normal unit layer, and a detection head unit composed of ordinary convolution with a relu activation function. Among them, the size and number of channels of the image input to the normal unit layer will not change, which can be understood as the feature dimension has not changed; the size of the image input to the reduced unit layer is reduced by half, and the number of channels is doubled.

[0104] As an option in the embodiment of the present application, training the first neural network based on the first loss value includes:

[0105] Fixing weight parameters of the first neural network and updating architecture parameters of the first neural network based on the first loss value;

[0106] After the architecture parameters of the first neural network are updated, the sample synthetic aperture radar image is re-input into the updated first neural network, and a second loss value is calculated based on the prediction result of the updated first neural network and the true result of the sample;

[0107] The architecture parameters of the updated first neural network are fixed, and the updated first neural network is updated again based on the second loss value.

[0108] Specifically, during the training of the first neural network, the weight parameters of the first neural network (which can also be understood as the parameters of all convolution operations themselves) can be fixed, and the architectural parameters of the first neural network can be updated based on the first loss value. The architectural parameters of the first neural network can be understood as the weight distribution of each edge and the weight distribution of each edge between intermediate nodes in the first neural network. Next, the sample synthetic aperture radar image can be input into the first neural network with the updated architectural parameters, and the second loss value can be calculated based on the prediction results of the updated first neural network and the actual sample results. The calculation method of the second loss value can be referred to in the above embodiment and will not be detailed here.

[0109] Next, the updated architecture parameters of the first neural network can be fixed, and the weight parameters of the first neural network can be updated based on the second loss value, thereby completing the first round of training of the first neural network.

[0110] It is understandable that in the embodiment of the present application, a training round number threshold can be pre-set, and until the number of training rounds of the first neural network reaches the set training round number threshold, it can be determined that the training of the first neural network is completed.

[0111] Step 108: Input the unprocessed synthetic aperture radar image into the second neural network, and display a target detection frame on the unprocessed synthetic aperture radar image based on the prediction result of the second neural network.

[0112] Specifically, after constructing the second neural network, the processed synthetic aperture radar image can be input into the second neural network to obtain corresponding prediction results. The prediction results may include, but are not limited to, the horizontal and vertical coordinates of the center points, offsets, and confidence levels of all target detection boxes in the processed synthetic aperture radar image. It should be noted that the prediction results of the second neural network may not yet meet the required accuracy, and this can be further optimized.

[0113] As another optional embodiment of the present application, displaying a target detection frame on the synthetic aperture radar image to be processed according to the prediction result of the second neural network includes:

[0114] Filter the target detection frames whose confidence is lower than the preset confidence threshold;

[0115] Perform non-maximum suppression on all filtered target detection frames;

[0116] All target detection frames that have undergone non-maximum suppression processing, the horizontal and vertical coordinates of the center point corresponding to each target detection frame, the offset, and the confidence level are displayed on the synthetic aperture radar image to be processed.

[0117] Specifically, all target detection frames in the prediction results of the second neural network whose confidence levels are lower than a preset confidence threshold can be filtered. This filtering method can be, but is not limited to, directly deleting the corresponding target detection frames, horizontal and vertical coordinates of the center point, offsets, and confidence levels.

[0118] Next, non-maximum suppression processing can be performed on all target detection frames after filtering. The non-maximum suppression processing can be used to remove redundant target detection frames in a certain area and retain the best target detection frame. The specific processing method is a technical means well known to those skilled in the art and will not be elaborated here.

[0119] Also see here Figure 4 FIG. 1 shows a schematic diagram of target detection in a synthetic aperture radar image provided by an embodiment of the present application. Figure 4 As shown, the best target detection box, the horizontal and vertical coordinates of the center point, the offset and the confidence level are displayed in each area of the synthetic aperture radar image.

[0120] As another option of the embodiment of the present application, before inputting the synthetic aperture radar image to be processed into the second neural network, the method further includes:

[0121] Inputting the sample synthetic aperture radar image into the second neural network, and calculating a third loss value based on the prediction result of the second neural network and the true result of the sample;

[0122] training the second neural network based on the third loss value;

[0123] Inputting the synthetic aperture radar image to be processed into the second neural network includes:

[0124] The synthetic aperture radar image to be processed is input into the trained second neural network.

[0125] Specifically, after constructing the second neural network, the second neural network can be trained using the above-mentioned sample synthetic aperture radar images and sample real results. The training method can refer to the above-mentioned method for training the first neural network, which will not be elaborated here.

[0126] See also Figure 5 , Figure 5 A schematic structural diagram of a target detection device for synthetic aperture radar images provided in an embodiment of the present application is shown.

[0127] like Figure 5 As shown, the target detection device for synthetic aperture radar images may include at least a first network construction module 501, a loss calculation module 502, a second network construction module 503, and a result display module 504, wherein:

[0128] A first network construction module 501 is configured to determine at least two convolution operations and construct a first neural network based on the at least two convolution operations;

[0129] a loss calculation module 502 for inputting the sample synthetic aperture radar image into the first neural network and calculating a first loss value based on the prediction result of the first neural network and the actual result of the sample;

[0130] A second network construction module 503 is configured to train the first neural network based on the first loss value, and derive a second neural network according to the architecture parameters of the trained first neural network and a preset derivation rule;

[0131] The result display module 504 is used to input the SAR image to be processed into the second neural network and display the target detection frame on the SAR image to be processed according to the prediction result of the second neural network.

[0132] In some possible embodiments, the first neural network includes two stem layers and eight unit layers, wherein:

[0133] The nth unit layer includes a first preprocessing node, a second preprocessing node, and a first intermediate node, a second intermediate node, a third intermediate node, and a fourth intermediate node arranged in sequence, the input end of the first preprocessing node is connected to the output end of the (n-2)th unit layer, and the output end of the first preprocessing node is connected to the input end of the first intermediate node, the input end of the second intermediate node, the input end of the third intermediate node, and the input end of the fourth intermediate node, respectively;

[0134] The input end of the second preprocessing node is connected to the output end of the (n-1)th unit layer, and the output end of the second preprocessing node is connected to the input end of the first intermediate node, the input end of the second intermediate node, the input end of the third intermediate node, and the input end of the fourth intermediate node respectively;

[0135] The output terminal of the first intermediate node is connected to the input terminal of the second intermediate node, the input terminal of the third intermediate node, and the input terminal of the fourth intermediate node respectively;

[0136] The output terminal of the second intermediate node is connected to the input terminal of the third intermediate node and the input terminal of the fourth intermediate node respectively;

[0137] An output terminal of the third intermediate node is connected to an input terminal of the fourth intermediate node;

[0138] The output end of the nth unit layer is used to splice the output results of the first intermediate node, the output results of the second intermediate node, the output results of the third intermediate node, and the output results of the fourth intermediate node, and output the processed output results to the second preprocessing node of the n+1th unit layer, where n is a positive integer greater than or equal to 3.

[0139] In some possible embodiments, the two stem layers include a first stem layer and a second stem layer, wherein:

[0140] The input end of the first preprocessing node in the first unit layer is connected to the output end of the first stem layer, and the input end of the second preprocessing node in the first unit layer is connected to the output end of the second stem layer;

[0141] The input end of the first preprocessing node in the second unit layer is connected to the output end of the second stem layer, and the input end of the second preprocessing node in the second unit layer is connected to the output end of the first unit layer.

[0142] In some possible embodiments, the second network construction module includes:

[0143] Fixing weight parameters of the first neural network and updating architecture parameters of the first neural network based on the first loss value;

[0144] After the architecture parameters of the first neural network are updated, the sample synthetic aperture radar image is re-input into the updated first neural network, and a second loss value is calculated based on the prediction result of the updated first neural network and the true result of the sample;

[0145] The architecture parameters of the updated first neural network are fixed, and the updated first neural network is updated again based on the second loss value.

[0146] In some possible embodiments, the device further includes:

[0147] Before inputting the to-be-processed synthetic aperture radar image into the second neural network, inputting the sample synthetic aperture radar image into the second neural network, and calculating a third loss value based on a prediction result of the second neural network and a true result of the sample;

[0148] training the second neural network based on the third loss value;

[0149] Inputting the synthetic aperture radar image to be processed into the second neural network includes:

[0150] The synthetic aperture radar image to be processed is input into the trained second neural network.

[0151] In some possible embodiments, the prediction results of the second neural network include the horizontal and vertical coordinates, offsets, and confidence levels of the center points of all target detection boxes in the synthetic aperture radar image to be processed;

[0152] The result display module includes:

[0153] Filter the target detection frames whose confidence is lower than the preset confidence threshold;

[0154] Perform non-maximum suppression on all filtered target detection frames;

[0155] All target detection frames that have undergone non-maximum suppression processing, the horizontal and vertical coordinates of the center point corresponding to each target detection frame, the offset, and the confidence level are displayed on the synthetic aperture radar image to be processed.

[0156] In some possible embodiments, the second neural network includes two stem layers and fourteen unit layers.

[0157] Those skilled in the art will clearly understand that the technical solutions of the embodiments of the present application can be implemented with the help of software and / or hardware. "Unit" and "module" in this specification refer to software and / or hardware that can independently perform or cooperate with other components to perform specific functions, where the hardware can be, for example, a field-programmable gate array (FPGA) or an integrated circuit (IC).

[0158] See also Figure 6 , Figure 6 A schematic structural diagram of another target detection device for synthetic aperture radar images provided in an embodiment of the present application is shown.

[0159] like Figure 6 As shown, the target detection device 600 for synthetic aperture radar images may include at least one processor 601 , at least one network interface 604 , a user interface 603 , a memory 605 , and at least one communication bus 602 .

[0160] The communication bus 602 may be used to implement connection and communication among the above components.

[0161] The user interface 603 may include buttons, and the optional user interface may also include a standard wired interface or a wireless interface.

[0162] The network interface 604 may include, but is not limited to, a Bluetooth module, an NFC module, a Wi-Fi module, and the like.

[0163] Among them, the processor 601 may include one or more processing cores. The processor 601 uses various interfaces and lines to connect the various parts of the entire electronic device 600, and executes various functions of the routing device 600 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 605, and calling data stored in the memory 605. Optionally, the processor 601 can be implemented in at least one hardware form of DSP, FPGA, PLA. The processor 601 can integrate one or a combination of CPU, GPU and modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to handle wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 601, but may be implemented separately through a chip.

[0164] The memory 605 may include RAM or ROM. Optionally, the memory 605 includes a non-transitory computer-readable medium. The memory 605 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 605 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 605 may also be optionally at least one storage device located away from the aforementioned processor 601. As Figure 6 As shown, the memory 605 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a target detection application for synthetic aperture radar images.

[0165] Specifically, the processor 601 may be configured to call a target detection application for synthetic aperture radar images stored in the memory 605 and specifically perform the following operations:

[0166] Determining at least two convolution operations, and constructing a first neural network based on the at least two convolution operations;

[0167] Inputting the sample synthetic aperture radar image into the first neural network, and calculating a first loss value based on the prediction result of the first neural network and the actual result of the sample;

[0168] Training the first neural network based on the first loss value, and deriving the second neural network according to the architecture parameters of the trained first neural network and a preset derivation rule;

[0169] The synthetic aperture radar image to be processed is input into the second neural network, and a target detection frame is displayed on the synthetic aperture radar image to be processed according to the prediction result of the second neural network.

[0170] In some possible embodiments, the first neural network includes two stem layers and eight unit layers, wherein:

[0171] The nth unit layer includes a first preprocessing node, a second preprocessing node, and a first intermediate node, a second intermediate node, a third intermediate node, and a fourth intermediate node arranged in sequence, the input end of the first preprocessing node is connected to the output end of the (n-2)th unit layer, and the output end of the first preprocessing node is connected to the input end of the first intermediate node, the input end of the second intermediate node, the input end of the third intermediate node, and the input end of the fourth intermediate node, respectively;

[0172] The input end of the second preprocessing node is connected to the output end of the (n-1)th unit layer, and the output end of the second preprocessing node is connected to the input end of the first intermediate node, the input end of the second intermediate node, the input end of the third intermediate node, and the input end of the fourth intermediate node respectively;

[0173] The output terminal of the first intermediate node is connected to the input terminal of the second intermediate node, the input terminal of the third intermediate node, and the input terminal of the fourth intermediate node respectively;

[0174] The output terminal of the second intermediate node is connected to the input terminal of the third intermediate node and the input terminal of the fourth intermediate node respectively;

[0175] An output terminal of the third intermediate node is connected to an input terminal of the fourth intermediate node;

[0176] The output end of the nth unit layer is used to splice the output results of the first intermediate node, the output results of the second intermediate node, the output results of the third intermediate node, and the output results of the fourth intermediate node, and output the processed output results to the second preprocessing node of the n+1th unit layer, where n is a positive integer greater than or equal to 3.

[0177] In some possible embodiments, the two stem layers include a first stem layer and a second stem layer, wherein:

[0178] The input end of the first preprocessing node in the first unit layer is connected to the output end of the first stem layer, and the input end of the second preprocessing node in the first unit layer is connected to the output end of the second stem layer;

[0179] The input end of the first preprocessing node in the second unit layer is connected to the output end of the second stem layer, and the input end of the second preprocessing node in the second unit layer is connected to the output end of the first unit layer.

[0180] In some possible embodiments, training the first neural network based on the first loss value includes:

[0181] Fixing weight parameters of the first neural network and updating architecture parameters of the first neural network based on the first loss value;

[0182] After the architecture parameters of the first neural network are updated, the sample synthetic aperture radar image is re-input into the updated first neural network, and a second loss value is calculated based on the prediction result of the updated first neural network and the true result of the sample;

[0183] The architecture parameters of the updated first neural network are fixed, and the updated first neural network is updated again based on the second loss value.

[0184] In some possible embodiments, before inputting the synthetic aperture radar image to be processed into the second neural network, the method further includes:

[0185] Inputting the sample synthetic aperture radar image into the second neural network, and calculating a third loss value based on the prediction result of the second neural network and the true result of the sample;

[0186] training the second neural network based on the third loss value;

[0187] Inputting the synthetic aperture radar image to be processed into the second neural network includes:

[0188] The synthetic aperture radar image to be processed is input into the trained second neural network.

[0189] In some possible embodiments, the prediction results of the second neural network include the horizontal and vertical coordinates, offsets, and confidence levels of the center points of all target detection boxes in the synthetic aperture radar image to be processed;

[0190] Based on the prediction results of the second neural network, a target detection frame is displayed on the synthetic aperture radar image to be processed, including:

[0191] Filter the target detection frames whose confidence is lower than the preset confidence threshold;

[0192] Perform non-maximum suppression on all filtered target detection frames;

[0193] All target detection frames that have undergone non-maximum suppression processing, the horizontal and vertical coordinates of the center point corresponding to each target detection frame, the offset, and the confidence level are displayed on the synthetic aperture radar image to be processed.

[0194] In some possible embodiments, the second neural network includes two stem layers and fourteen unit layers.

[0195] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above method. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a microdrive, a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0196] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0197] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0198] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interface, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0199] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0200] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0201] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, magnetic disk, or optical disk, etc., various media that can store program code.

[0202] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program. The program may be stored in a computer-readable memory, which may include a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0203] The above are merely exemplary embodiments of the present disclosure and are not intended to limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. After considering the specification and practicing the disclosure herein, those skilled in the art will easily think of the implementation scheme of the present disclosure. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not recorded in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A target detection method for synthetic aperture radar images, characterized in that: include: Determining at least two convolution operations, and constructing a first neural network based on the at least two convolution operations; Inputting the sample synthetic aperture radar image into the first neural network, and calculating a first loss value based on the prediction result of the first neural network and the sample actual result; Training the first neural network based on the first loss value, and deriving a second neural network according to architecture parameters in the trained first neural network and preset derivation rules; The synthetic aperture radar image to be processed is input into the second neural network, and a target detection frame is displayed on the synthetic aperture radar image to be processed according to the prediction result of the second neural network.

2. The method according to claim 1, characterized in that The first neural network includes two stem layers and eight unit layers, wherein: The nth unit layer includes a first preprocessing node, a second preprocessing node, and a first intermediate node, a second intermediate node, a third intermediate node, and a fourth intermediate node arranged in sequence, wherein the input end of the first preprocessing node is connected to the output end of the (n-2)th unit layer, and the output end of the first preprocessing node is connected to the input end of the first intermediate node, the input end of the second intermediate node, the input end of the third intermediate node, and the input end of the fourth intermediate node, respectively; The input end of the second preprocessing node is connected to the output end of the (n-1)th unit layer, and the output end of the second preprocessing node is connected to the input end of the first intermediate node, the input end of the second intermediate node, the input end of the third intermediate node, and the input end of the fourth intermediate node respectively; The output terminal of the first intermediate node is connected to the input terminal of the second intermediate node, the input terminal of the third intermediate node, and the input terminal of the fourth intermediate node respectively; The output terminal of the second intermediate node is connected to the input terminal of the third intermediate node and the input terminal of the fourth intermediate node respectively; The output end of the third intermediate node is connected to the input end of the fourth intermediate node; The output end of the nth unit layer is used to splice the output result of the first intermediate node, the output result of the second intermediate node, the output result of the third intermediate node and the output result of the fourth intermediate node, and output the processed output result to the second preprocessing node of the n+1th unit layer, where n is a positive integer greater than or equal to 3.

3. The method according to claim 2, characterized in that The two stem layers include a first stem layer and a second stem layer, wherein: The input end of the first preprocessing node in the first unit layer is connected to the output end of the first stem layer, and the input end of the second preprocessing node in the first unit layer is connected to the output end of the second stem layer; The input end of the first preprocessing node in the second unit layer is connected to the output end of the second stem layer, and the input end of the second preprocessing node in the second unit layer is connected to the output end of the first unit layer.

4. The method according to claim 1, wherein The training of the first neural network based on the first loss value includes: Fixing weight parameters of the first neural network and updating architecture parameters of the first neural network based on the first loss value; After the architecture parameters of the first neural network are updated, the sample synthetic aperture radar image is re-inputted into the updated first neural network, and a second loss value is calculated based on the prediction result of the updated first neural network and the true result of the sample; Fix the updated architecture parameters of the first neural network, and update the updated first neural network again based on the second loss value.

5. The method according to claim 1, wherein Before inputting the synthetic aperture radar image to be processed into the second neural network, the method further includes: Inputting the sample synthetic aperture radar image into the second neural network, and calculating a third loss value based on a prediction result of the second neural network and a true result of the sample; Training the second neural network based on the third loss value; Inputting the synthetic aperture radar image to be processed into the second neural network includes: The synthetic aperture radar image to be processed is input into the trained second neural network.

6. The method according to claim 5, characterized in that The prediction results of the second neural network include the horizontal and vertical coordinates, offsets and confidence levels of the center points of all target detection frames in the synthetic aperture radar image to be processed; The step of displaying a target detection frame on the synthetic aperture radar image to be processed according to the prediction result of the second neural network includes: Filtering the target detection frames whose confidence is lower than a preset confidence threshold; Performing non-maximum suppression processing on all the target detection frames that have been filtered; All the target detection frames that have undergone non-maximum suppression processing, the horizontal and vertical coordinates of the center point corresponding to each target detection frame, the offset, and the confidence level are displayed on the synthetic aperture radar image to be processed.

7. The method according to claim 2, characterized in that The second neural network includes two stem layers and fourteen unit layers.

8. A target detection device for synthetic aperture radar images, characterized in that: include: a first network construction module, configured to determine at least two convolution operations and construct a first neural network based on the at least two convolution operations; a loss calculation module, configured to input the sample synthetic aperture radar image into the first neural network and calculate a first loss value based on the prediction result of the first neural network and the sample actual result; a second network construction module, configured to train the first neural network based on the first loss value, and derive a second neural network according to architecture parameters of the trained first neural network and preset derivation rules; The result display module is used to input the synthetic aperture radar image to be processed into the second neural network and display the target detection frame on the synthetic aperture radar image to be processed according to the prediction result of the second neural network.

9. A target detection device for synthetic aperture radar images, characterized in that: including a processor and a memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes the steps of the method according to any one of claims 1 to 7.

Images