A SAR image target training and prediction method based on a unified network

Abstract

This application relates to the fields of deep learning and SAR image recognition and detection, and presents a SAR image target training and prediction method based on a unified network. It reduces training time by establishing a multi-type target training set of SAR images with a unified image size. Then, a deep learning network is trained on the multi-type training set to obtain a deep learning network and network weights applicable to all images to be detected. Next, the image to be monitored is divided into overlapping blocks, and each image sub-block is detected separately to obtain the corresponding target information. Finally, all image sub-blocks are merged, and the detected target information is combined to obtain the required target information set. This method offers fast detection speed and high detection accuracy.

Description

Technical Field

[0001] This application belongs to the field of deep learning and SAR image recognition and detection, and specifically relates to a method for SAR image target training and prediction based on a unified network. Background Technology

[0002] With the rapid development of convolutional neural network technology in the field of optical image target recognition, a large number of SAR image target recognition algorithms based on deep neural networks have emerged. However, there are the following problems in transplanting mature deep neural network algorithms from the optical field to SAR image target recognition engineering applications:

[0003] First, the resolution of SAR images is known, and the resolution of samples may be different. Therefore, resolution information can be used for preprocessing, and normalization processing is required for different resolutions.

[0004] Second, in engineering applications, SAR images are much larger than conventional optical images, and there are different sizes, which inevitably requires consideration of the impact of input image segmentation and result merging.

[0005] Third, SAR images show a wide range of target sizes, requiring consideration of handling targets with large-scale differences.

[0006] Therefore, considering the SAR image target recognition problem with different image resolutions, different image sizes, and different target types, a new training and prediction method is needed to reduce training time and increase recognition rate. Summary of the Invention

[0007] The purpose of this application is to provide a SAR image target training and prediction method based on a unified network to solve the problems of long training time and low recognition rate of SAR images in the prior art.

[0008] The technical solution of this application is: a SAR image target training and prediction method based on a unified network, comprising: establishing a SAR image multi-type target training set with a unified image size based on different types of SAR image databases; training the training set with a suitable deep learning network to obtain the training network weights; receiving images in real time, dividing the image to be detected into multiple image sub-blocks by overlap, recording the target parameters of each image sub-block, and performing prediction on each image sub-block using a deep learning network; merging and processing strategic target and tactical target images separately, and merging and processing the detected target information.

[0009] Preferably, the target training set generation method for different types of SAR image databases includes: unifying the image reference size; classifying the database images according to size into strategic target images and tactical target images, and processing the strategic target images and tactical target images separately; and labeling the position and type of the targets in all image samples in the training set to generate the target information set required for training.

[0010] Preferably, the method for processing the strategic target images and tactical target images includes: generating a strategic target training set using a random target method, randomly cropping sub-images of integer multiples of the reference size from the original image database to generate a first training set; generating a strategic target training set using a real target localization method, cropping sub-images of strategic targets of integer multiples of the reference size from the original image database, with the strategic target size in the sub-image occupying at least 1 / 3 of the entire image, to generate a second training set; generating a tactical target training set using a random target method, cropping sub-images of the same size as the reference from the original image database to generate a third training set; generating a tactical target training set using a real localization method, cropping sub-images of the same size as the reference from the original image database to generate a fourth training set; and performing smooth downsampling processing on all images in the first and second training sets to generate new first and second training sets, with the image size uniformly set to the reference size.

[0011] Preferably, the method for real-time detection of the image, which determines whether the target to be detected is a tactical or strategic target, sets segmentation parameters according to the different targets, reads network parameters, and includes receiving the image and dividing the image to be detected into overlapping blocks, wherein the sub-block image is a square and the overlap size of the sub-block image is greater than or equal to 1 / 8 of the sub-block image size; detecting each image sub-block according to a trained deep learning network; and normalizing the image sub-blocks to convert the sub-block target information into global image information.

[0012] Preferably, the method for normalizing the image sub-blocks includes converting the target location information of each image sub-block into global target location information; storing the target detection results of all image sub-blocks to obtain the target information set of the image.

[0013] Preferably, the specific method for detecting the image sub-blocks includes detecting the input of the image sub-blocks, the detection network, and the network training weights; processing each image sub-block through the forward propagation of the deep learning network; and establishing a target information set based on the target's position in the image sub-block, the target type, and the confidence level.

[0014] Preferably, when merging strategic objectives, if any two strategic objectives in the objective set are of the same type and their Intersection over Union (IOU) is greater than 10%, they are merged into one objective and added to the merged objective set. When merging tactical objectives, if any two tactical objectives in the objective set are of the same type and their IOU is greater than 60%, the objective with higher confidence is selected and added to the merged objective set, and the objective with lower confidence is deleted from the source objective information set.

[0015] Preferably, the specific steps of the target merging process are as follows: initialize the merged target set and set the number of targets in the target set to 0; cyclically query the source target information set, obtain the target type parameter, compare the target with each target in the merged target set, and execute the corresponding target merging strategy according to the target type parameter; if the target queried from the source target information set does not have a related target that can be merged in the merged target set, then add the target to the merged target set; cycle through the source target information set to generate a new merged target information set.

[0016] This application presents a SAR image target training and prediction method based on a unified network. It reduces training time by establishing a multi-type target training set of SAR images with a unified image size. Then, it trains a deep learning network on the multi-type training set to obtain a deep learning network and network weights applicable to all images to be detected. Next, it divides the image to be monitored into overlapping blocks, detects each image sub-block separately to obtain the corresponding target information, and finally merges all image sub-blocks and the detected target information to obtain the required target information set. This method offers fast detection speed and high detection accuracy. Attached Figure Description

[0017] To more clearly illustrate the technical solutions provided in this application, the accompanying drawings will be briefly described below. Obviously, the drawings described below are merely some embodiments of this application.

[0018] Figure 1 This is a schematic diagram of the overall process of this application;

[0019] Figure 2 A flowchart illustrating the process of establishing the training set for this application;

[0020] Figure 3 This is a schematic diagram of the training process for the image to be detected in this application;

[0021] Figure 4 This is a schematic diagram of the image sub-block merging process in this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be described in more detail below with reference to the accompanying drawings.

[0023] A method for SAR image target training and prediction based on a unified network, such as Figure 1 , Figure 2 As shown, it includes the following steps:

[0024] Step S100: Based on different types of SAR image databases, establish a SAR image training set with a uniform image size for multiple target types;

[0025] The specific training methods include:

[0026] Step S110: Unify the image baseline size N_Base×N_Base. The baseline size serves as the standard size for all images. In this method, the image sample size in the training set is uniformly N_Base×N_Base (N_Base is adjustable and must be an integer multiple of 32).

[0027] Step S120: Classify the database images according to size into strategic target images and tactical target images. Strategic target images are large-sized images, while tactical target images are small-sized images. Since the resolutions of the two are different, strategic target images and tactical target images need to be processed separately.

[0028] Since the number of samples for the same target varies, it is difficult to obtain satisfactory training results by directly locating the target for a small number of samples. Therefore, for image databases with a small number of samples, random location is used to increase the number of samples in order to obtain satisfactory training results.

[0029] For strategic target image databases with a small number of samples, a strategic target training set is generated using a random target method. Sub-images that are random multiples of the baseline size are cropped from the original image database to generate the first training set.

[0030] For strategic target image databases with a large number of samples, a strategic target training set is generated using the real target localization method. Within the original image database, sub-images of strategic targets are cropped to integer multiples of the baseline size, and the size of the strategic targets in the sub-images occupies at least 1 / 3 of the entire image, to generate a second training set.

[0031] For a tactical target image database with a small number of samples, a tactical target training set is generated by using a random target method. A sub-image of the same size as the baseline is captured from the original image database to generate a third training set.

[0032] For a tactical target image database with a large number of samples, a tactical target training set is generated using a real-world positioning method. A sub-image with the same reference size is cropped from the original image database to generate a fourth training set.

[0033] All images in the first and second training sets are subjected to smooth downsampling to generate new first and second training sets. The image size is uniformly set to the baseline size N_Base×N_Base.

[0034] Step S130: The location and type of the target are labeled for all image samples in the training set to generate the target information set required for training. By unifying the graphic size and then processing it, the image reading and image preprocessing time in the training process can be reduced to a large extent, thus greatly reducing the training time of the entire training set.

[0035] Step S200: Train a suitable deep learning network on the training set to obtain the training network weights;

[0036] Deep learning-based recognition and detection networks need to meet the following conditions:

[0037] 1. Real-time performance requires a comprehensive consideration of both computational performance and detection accuracy;

[0038] 2. Universality of network architecture: It possesses the mainstream processing architecture of current deep learning, including conventional processing layers such as residual structure, batch normalization, and convolution. At the same time, the network architecture is not a heterogeneous architecture, which introduces uncertainties to training, prediction, and software implementation.

[0039] 3. Multi-scale: Considering the size differences of the same type of SAR targets in different datasets, the training network needs to have a multi-scale architecture design.

[0040] 4. Cluster small target detection: The deep recognition and detection network needs to have good cluster small target detection capabilities.

[0041] Based on the above considerations, the deep learning network is trained using a deep learning network based on YOLOv3 or later.

[0042] Step S300: Receive images in real time, divide the image to be detected into multiple overlapping image sub-blocks, predict each image sub-block using a deep learning network, and then record the target parameters of each image sub-block. Each image sub-block is trained using the same network weights. This step divides images in the same image database into uniform sizes, enabling unified detection of targets of different resolutions and types. The detection process is simplified and unified, achieving a good balance between real-time software performance and detection accuracy.

[0043] like Figure 3As shown, the specific methods include:

[0044] Step S310: Determine whether the target to be detected is a tactical target or a strategic target, set the block parameters according to the different targets, read the network parameters, receive the image and perform overlapping block division on the image to be detected;

[0045] 1. For strategic target detection, the image to be detected is divided into overlapping blocks. The sub-block image is a square with a size >= 2048; (2048 can be adjusted and is adaptively determined according to the size of the whole image in actual design).

[0046] 2. For tactical target detection, the image to be detected is divided into overlapping blocks, and the sub-block image is a square with a size of N_base;

[0047] 3. The overlap size of the sub-image is greater than or equal to 1 / 8 of the size of the sub-block image.

[0048] After overlapping and segmenting, record the position, type information, etc. of each image sub-block.

[0049] Step S320: Image sub-blocks are read cyclically. Each sub-block is detected using the deep learning network trained in step S200. A uniform network weight is used for each sub-block. Because a uniform approach is used for each sub-block, detection efficiency is significantly improved. Specifically, this includes:

[0050] Input detection: Detect the input of image sub-blocks, the detection network, and the network training weights;

[0051] Detection and processing: Each image sub-block is processed through forward propagation of a deep learning network;

[0052] Detection output: A target information set is established by the target's position in the image sub-block, target type, and confidence level.

[0053] Step S330: After each image sub-block is detected, the image sub-block is normalized. The target parameter information of the image sub-block recorded in step S310 is normalized to ensure that all image sub-blocks can be accurately merged. Specifically, the target position information of each image sub-block is converted into global target position information; the target detection results of all image sub-blocks are stored to obtain the target information set of the image.

[0054] Step S400: Merge the strategic and tactical target images separately, and merge the detected target information.

[0055] In the case of image overlap segmentation, the overlap ratio of image sub-blocks of strategic targets and tactical targets is different. The overlap ratio of strategic targets is smaller, while that of tactical targets is larger. Therefore, when merging, the overlap of tactical targets is larger.

[0056] When merging strategic objectives, if any two strategic objectives in the objective set are of the same type and their Intersection over Union (IOU) is greater than 10%, they are merged into one objective and added to the merged objective set.

[0057] When merging tactical targets, if any two tactical targets in the target set are of the same type and their Intersection over Union (IOU) is greater than 60%, then the target with the higher confidence level is selected to be added to the merged target set, and the target with the lower confidence level is deleted from the source target information set.

[0058] like Figure 4 As shown, regardless of whether it is a strategic or tactical objective, the specific steps for objective merging are as follows:

[0059] Step S410: Initialize the merged target set and set the number of the target set to 0;

[0060] Step S420: Loop through the source target information set to obtain the target type parameter, compare the target with each target in the merged target set, and execute the corresponding target merging strategy according to the target type parameter;

[0061] Step S430: If the target queried from the source target information set does not have a related target that can be merged in the merged target set, then add the target to the merged target set.

[0062] Step S440: Complete the round-robin process of the source target information set and generate a new target information set after merging.

[0063] By initializing the merged target set, each image sub-block within the target set can be merged quickly in a unified manner, thereby ensuring the efficiency of the merging process.

[0064] This application has the following advantages:

[0065] 1. Reduce training time: SAR images have varying reference sizes and are much larger than regular images. Conventional methods often result in excessively long training times, significantly increasing the learning cost of effective information from the SAR database. This method normalizes training images of different target types to a smaller size, greatly reducing image reading and preprocessing time during training, thus drastically reducing the overall training time.

[0066] 2. Unified detection framework: This method uses the same type of deep learning network to train different target types after normalization during the training process to obtain unified network weights after training. Then, through a unified overlapping block detection strategy, it can complete the unified detection of different types of SAR targets at different resolutions. The detection process is simplified and unified. Although the block strategy increases the detection time, the addition of different block size settings for strategic and tactical purposes can achieve a good balance between software real-time performance and detection accuracy.

[0067] 3. Improve the accuracy of multi-type target recognition and detection in different scenarios of SAR platform. The samples in SAR database usually come from various airborne or spaceborne platforms, and have the characteristics of size diversity and wide mapping band. The data size and diversity are much larger than the conventional training samples collected by optical cameras. This method solves the problem of detection accuracy of small tactical targets by overlapping blocks. For large strategic targets with few samples, random size splitting is adopted in the limited training sample set to artificially generate multi-size sample data of a certain type of strategic target, so as to meet the sample effectiveness of training and detection.

[0068] This method can be applied to a unified target recognition and detection process for SAR with different resolutions, image sizes, and target types. It reduces the complexity of training and detection processes for multiple types of SAR targets, improves recognition and detection accuracy, and can serve as an engineering implementation method for real-time interpretation of SAR images by future UAVs or manned aircraft.

[0069] As a specific implementation method, the following example illustrates the specific measures taken:

[0070] - The training set currently includes two types of small targets: airport targets and vehicle targets.

[0071] The baseline size of the training set is 512.

[0072] The airport image training set is obtained by randomly selecting a fixed size of 4096 cubes from the original image according to its resolution, and then smoothing the extraction to uniformly reduce the size to 512 cubes.

[0073] The training set of car images is a set of squares of a fixed size of 512 randomly selected from the original image.

[0074] - The same architecture network is used to train all target sets, and the same trained network is used for real-time prediction.

[0075] - Image overlapping block strategy

[0076] Large target detection: Image sub-block size is 4096, overlap size is 512;

[0077] Small object detection: Image sub-block size is 512, and overlap size is 64;

[0078] -Detection Strategy

[0079] Real-time detection using two block-segmentation strategies is performed simultaneously on the original image.

[0080] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for training and predicting SAR image targets based on a unified network, characterized in that: include, Based on different types of SAR image databases, a training set of SAR images with a uniform image size and multiple target types is established; Train a suitable deep learning network on the training set to obtain the training network weights; The system receives images in real time, divides the image to be detected into multiple overlapping sub-blocks, records the target parameters of each sub-block, and uses a deep learning network to predict each sub-block. The images of strategic and tactical targets are merged and processed separately, and the detected target information is then merged and processed. Methods for generating target training sets for different types of SAR image databases include: Standardize image reference size; The database images are classified according to size into strategic target images and tactical target images, and then processed separately for each type of image. The location and type of the target are labeled for all image samples in the training set to generate the target information set required for training. The methods for processing the strategic target images and tactical target images include, A strategic target training set is generated using a random target approach. Sub-images of integer multiples of the baseline size are randomly cropped from the original image database to generate the first training set. A strategic target training set is generated using a real target localization method. Within the original image database, a sub-image of the strategic target is cropped to an integer multiple of the baseline size. The strategic target size in the sub-image occupies at least 1 / 3 of the entire image, and a second training set is generated. A tactical target training set is generated using a random target method. A sub-image of the same size as the baseline is captured from the original image database to generate a third training set. A tactical target training set is generated using a real-world positioning method. A sub-image of the same size as the baseline is cropped from the original image database to generate a fourth training set. All images in the first and second training sets are subjected to smooth downsampling to generate new first and second training sets, with the image size uniformly set to the baseline size. The method for real-time detection of the image includes, The system determines whether the target to be detected is a tactical or strategic target, sets the block parameters according to the target, reads the network parameters, receives the image, and divides the image to be detected into overlapping blocks. The sub-block image is square, and the overlap size of the sub-block image is greater than or equal to 1 / 8 of the sub-block image size. Each image sub-block is detected using a pre-trained deep learning network; The image sub-blocks are normalized to transform the target information of the sub-blocks into global image information.

2. The SAR image target training and prediction method based on a unified network as described in claim 1, characterized in that: The method for normalizing the image sub-blocks includes, Transform the target location information of each image sub-block into global target location information; Store the target detection results of all image sub-blocks to obtain the target information set of the image.

3. The SAR image target training and prediction method based on a unified network as described in claim 1, characterized in that: The specific methods for detecting the image sub-blocks include, The input to image sub-blocks, the detection network, and the network training weights is detected; Each image sub-block is processed through forward propagation of a deep learning network; A target information set is established by considering the target's location in an image sub-block, its type, and its confidence level.

4. The SAR image target training and prediction method based on a unified network as described in claim 1, characterized in that: When merging strategic objectives, if any two strategic objectives in the objective set are of the same type and their Intersection over Union (IOU) is greater than 10%, they are merged into one objective and added to the merged objective set. When merging tactical targets, if any two tactical targets in the target set are of the same type and their Intersection over Union (IOU) is greater than 60%, then the target with the higher confidence level is selected to be added to the merged target set, and the target with the lower confidence level is deleted from the source target information set.

5. The SAR image target training and prediction method based on a unified network as described in claim 4, characterized in that: The specific steps for target merging are as follows: Initialize the merged target set and set the size of the target set to 0; Loop through the source target information set to obtain the target type parameter, compare the target with each target in the merged target set, and execute the corresponding target merging strategy according to the target type parameter; If the target found in the source target information set is not found to be a related target that can be merged in the merged target set, then the target is added to the merged target set. The source target information set is completed by round-robin, and a new target information set is generated after merging.

Images