Network model extension based method for identifying newly added SAR target

Abstract

The application discloses a new-class SAR target recognition method based on network model expansion, takes a YOLOv7 network as a benchmark model, expands a convolution module in the model, saves parameters of the original module unchanged, trains the expanded module using a new-class target, and reorganizes the structure after the training is completed, so that the number of parameters of the model is not increased with the increase of the classes, and a class confusion loss function is added in the new-class training, the direct distinction between new and old classes is increased, and the confusion of similar classes is reduced.

Description

Technical Field

[0001] This invention belongs to the field of radar technology, specifically relating to a new type of SAR target recognition method based on network model extension. Background Technology

[0002] Synthetic Aperture Radar (SAR) is a system that actively observes and images targets. Operating in the microwave band, SAR systems are unaffected by weather and lighting conditions, providing all-weather, all-time Earth observation capabilities. With the continuous expansion of SAR system applications, the observed objects, mission objectives, and recognition requirements are showing a significant trend of diversification. Traditional SAR recognition systems are typically designed for single tasks, with a fixed set of categories and clearly defined task boundaries. However, SAR target recognition systems are often required to simultaneously complete multiple types of perception and recognition tasks under multi-task, multi-time period, and multi-scale conditions, forming a cross-task, cross-domain composite recognition system. This diversification trend directly leads to a continuous expansion of the target categories that the system needs to recognize, thus creating an urgent need for a dynamic category update mechanism. In deep network model target recognition tasks, the data distribution for incremental learning is not fixed. If the trained model is directly used to solve the incremental learning problem, a "catastrophic forgetting" problem will occur. Furthermore, when new categories contain data similar to those of old categories, the new and old categories are prone to overlap. Therefore, it is necessary to optimize the target recognition model to achieve higher accuracy in recognizing new targets in SAR images.

[0003] The current mainstream approach for identifying newly added target classes is to merge all available samples from both the old and new classes into a single "full" training set, and then retrain the entire deep network in an end-to-end manner. This process typically involves a warm start of the old model weights, followed by multiple rounds of fine-tuning on the mixed old and new data. This allows the network to retain the discrimination boundaries of the old classes while fitting the feature distribution of the new classes. After training, hyperparameters are fine-tuned using a joint validation set, and the model is redeployed. This method is logically simple, mature in implementation, and can directly reuse existing deep learning frameworks, making it the default option for industrial deployment. However, the cost is that each time a new class is added, all old class data must be used, and the storage and training time increases linearly with the number of classes. Existing incremental learning methods can maintain the recognition performance of the old classes while training only with new class data, but the confusion between the old and new classes still exists. In target classification, the confusion is usually reduced by explicitly expanding the decision boundaries of different classes in the feature space, which provides a new approach for identifying newly added target classes in SAR. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, this invention provides a new type of SAR target recognition method based on network model extension. Using the YOLOv7 network as the baseline model, the convolutional modules in the model are extended while keeping the parameters of the original modules unchanged. The extended modules are trained using new target categories, and the structure is reorganized after training to ensure that the number of model parameters does not increase with the increase of categories. Furthermore, a category confusion loss function is added during the training of new categories to increase the discriminative power between new and old categories and reduce confusion between similar categories.

[0005] The technical solution adopted by this invention to solve its technical problem is as follows: Step 1: Dataset Construction; Select SAR images containing the targets to be identified, and label the targets in the images. Crop the images to the size that meets the model input requirements to obtain the labeling results of the corresponding slice images. Before the network training begins, randomly divide all training samples into training set, validation set and test set according to the proportion. Step 2: Target recognition network model construction; The YOLOv7 network model is improved by two parts: expanding the convolutional modules and adding a confusion loss function. At the original kernel size The convolutional module is embedded in parallel with another kernel of size 1. The convolutional module is expanded to have the same number of input and output channels as the original convolutional module, and the parameters of the original convolutional module are used to initialize the new convolutional kernel; when training the model using new class data, the parameters of the original convolutional module are frozen, and only the parameters of the new convolutional module are iterated. The original convolutional module is represented as: (1) in The output feature map, These are the weights of the original convolutional module. For bias; The extended convolutional module is represented as: (2) in To expand the weights of the convolutional module, For bias; Convolution result after convolution module expansion for: (3) In convolution processing with a kernel size of 1 branch, the padding size is adjusted to ensure that the feature map sizes are exactly the same; A class confusion loss function is added to the YOLOv7 loss function to expand the decision boundaries of different classes in the feature space; new classes are included. The feature map obtained by feature extraction from the image is In the detector In the branch, assume The target is identified as In this case, the category information of the target label is temporarily changed manually. And calculate the detection loss. New category Compared to the old category The higher the similarity, the greater the loss. The smaller; in the detector In the branch, Perform normal target detection and calculate the detection loss. Let the total classification loss of this training be... ,but: (4) in These are the weights of the loss function; Step 3: Network training; Input the training set and validation set into the target recognition network model built in step 2 for training and validation, respectively. When the loss and accuracy of the validation set converge, select and save the training model for the corresponding training round. Step 4: Network restructuring; After the model training is completed in step 3, the structure reparameterization technique is used to seamlessly integrate the expanded convolutional module in step 2 into the original convolutional module, so that the number of model parameters does not increase with the increase of categories. The weights of the convolutional kernels in the fusion single-branch model are , bias is The fused convolution result for: (5) An electronic device includes: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs the above-mentioned new type of SAR target recognition method.

[0006] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned new type of SAR target recognition method.

[0007] A chip includes a processor for calling and running a computer program from a memory, causing a device equipped with the chip to perform the aforementioned new type of SAR target recognition method.

[0008] A computer program product includes a computer storage medium storing a computer program, the computer program including instructions executable by at least one processor, which, when executed by the at least one processor, implement the aforementioned new type of SAR target recognition method.

[0009] The beneficial effects of this invention are as follows: This invention provides a new SAR target recognition method based on network model extension. Compared with traditional methods, this invention can overcome the dependence on old category data, train the module only using new category data, maintain the model's recognition performance for old categories, and distinguish similar categories in the old and new categories, effectively reducing the catastrophic forgetting and confusion between new and old target categories in the recognition of new SAR targets. Attached Figure Description

[0010] Figure 1 This is a flowchart of a new SAR target recognition method based on network model extension.

[0011] Figure 2 This is a schematic diagram of the expansion of the convolution module used in this invention.

[0012] Figure 3 This is a schematic diagram of the network structure reorganization used in this invention. Figure 3 (a) Original multi-branch model, Figure 3 (b) Integrating single-branch models.

[0013] Figure 4 This is a schematic diagram illustrating the training of the confusion loss function used in this invention.

[0014] Figure 5 The recognition results after adding a category confusion loss function. Detailed Implementation

[0015] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0016] This invention addresses the catastrophic forgetting and confusion between new and old target categories in SAR target identification by proposing a new target category identification method based on network model extension. This invention can improve the identification performance of new target categories while maintaining the recognition performance of old target categories, and effectively distinguish similar categories among new and old target categories, supporting the intelligent application of SAR systems.

[0017] The basic idea of ​​this invention is to use the YOLOv7 network as the baseline model, extend the convolutional module in the model, keep the parameters of the original module unchanged, train the extended module with the new category target, and restructure the structure after training to keep the number of model parameters from increasing with the increase of categories. In the training of the new category, a category confusion loss function is added to increase the discriminability between the old and new categories and reduce the confusion between similar categories.

[0018] like Figure 1 To achieve the above objectives, this invention provides a method for identifying new target classes based on network model extension, which mainly includes the following steps: Step 1: Dataset construction.

[0019] Select SAR images containing the targets to be identified, and label the targets in the images. Crop the images to the size that meets the model input requirements to obtain the labeling results of the corresponding slice images. Before the network training begins, randomly divide all training samples into training set, validation set and test set according to the proportion.

[0020] Step 2: Target recognition network model construction.

[0021] This invention improves the original YOLOv7 network model by expanding the convolutional module and adding a confusion loss function.

[0022] At the original kernel size The convolutional module is embedded in parallel with another kernel of size [missing information]. The new convolutional module has the same number of input and output channels as the original convolutional module, and its kernel is initialized using the parameters of the original convolutional module. When training the model with new class data, the parameters of the original convolutional module are frozen, and only the parameters of the new convolutional module are iterated.

[0023] The original convolutional module can be represented as: (1) in The output feature map, The weights of the original convolutional module, For bias.

[0024] The extended convolutional module can be represented as: (2) in To expand the weights of the convolutional module, For bias. The convolution result after the convolution module is expanded. for (3) The number of newly added convolutional channels is the same as the original number of convolutional channels, and the number of channels in the resulting convolution is also exactly the same. However, since the original convolutional kernel size is different from the newly added convolutional kernel size, the feature map size of each channel after the two convolutional processes differs for input feature maps of the same size. In order to ensure that the two convolutional results can be directly added, the padding size needs to be manually adjusted in the convolutional processing with a kernel size of 1 branch to ensure that the feature map size is exactly the same.

[0025] A class confusion loss function is added to the YOLOv7 loss function to expand the decision boundaries of different classes in the feature space. This includes new classes. The feature map obtained by feature extraction from the image is , in the detector In the branch, assume The target is identified as In this case, the category information of the target label is temporarily changed manually. And calculate the detection loss. New category Compared to the old category The higher the similarity, the greater the loss. The smaller; in the detector In the branch, Perform normal target detection and calculate the detection loss. Let the total classification loss in this training be... ,but (4) in These are the weights of the loss function.

[0026] Step 3: Network training.

[0027] Input the training set and validation set into the network built in step 2 for training and validation, respectively. When the loss and accuracy of the validation set converge, select and save the training model for the corresponding training round.

[0028] Step 4: Network restructuring.

[0029] After the model training is completed in step 3, the new model is seamlessly integrated into the old model using structural reparameterization, ensuring that the number of model parameters does not increase with the increase of categories.

[0030] The weights of the convolutional kernels in the fusion single-branch model are , bias is The fused convolution result for: (5) Step 5: Network test.

[0031] The test set is input into the target recognition model after structural reconstruction in step 4 for forward inference to obtain the corresponding prediction results. These prediction results are then compared with the true labels of the test set to test the target recognition performance of the model.

[0032] Example: Step 1: Dataset construction.

[0033] This example uses MSTAR data for dataset construction. MSTAR data contains various target slices and clutter background images. Ten target categories from the MSTAR data are selected as the research objects for the new target recognition problem. These ten target categories and the background image are then stitched together to obtain a large-scene SAR image. The large-scene image is then cropped to a size of [size missing]. An image of pixels. Assume the size of the large scene is... The total number of targets in the scene is m, and the set of all targets is m. The set of targets that have been cropped is The set of unpruned targets is ,gather satisfy: (6) in Represents the empty set. Randomly select a target. , The size is The central position in the large scene is Then a size is Pixels and can completely contain the target slice image The range of pixels in the top left corner of the large scene for: (7) in , , , The range of values ​​for is: (8) Determine Set All targets Is it included in the image? In the image In the middle, then Put into Otherwise Put into Repeat the above clipping process until the set is complete. .

[0034] This example divides the obtained 10-class dataset into multiple detection and recognition tasks to verify the new category recognition capability of the method used in this chapter. Table 1 shows the size of the training and test sets for the 10 target classes, as well as the method for setting the category increment.

[0035] Table 1 Incremental Settings for Target Categories

[0036] Step 2: Target recognition network model construction.

[0037] This example improves the original YOLOv7 network model by expanding the convolutional modules and adding a confusion loss function.

[0038] Figure 2 The convolutional module expansion method used in this example has an original kernel size of The convolutional module is embedded in parallel with another kernel of size [missing information]. The new convolutional module has the same number of input and output channels as the original convolutional module, and its kernel is initialized using the parameters of the original module. When training the model with new class data, the parameters of the original convolutional module are frozen, and only the parameters of the new convolutional module are iterated. The original convolutional module can be represented as: (9) in The output feature map, The weights of the original convolutional module, For bias. The extended convolutional module can be represented as: (10) in To expand the weights of the convolutional module, For bias. The convolution result after the convolution module is expanded. for: (11) The number of newly added convolutional channels is the same as the original number of convolutional channels, and the number of channels in the resulting convolution is also exactly the same. However, since the original convolutional kernel size is different from the newly added convolutional kernel size, the feature map size of each channel after the two convolutional processes differs for input feature maps of the same size. In order to ensure that the two convolutional results can be directly added, the padding size needs to be manually adjusted in the convolutional processing with a kernel size of 1 branch to ensure that the feature map size is exactly the same.

[0039] Figure 4 This is a diagram illustrating the training using the confusion loss function in this example. It adds a class confusion loss function to the YOLOv7 loss function, incorporating new classes. The feature map obtained by feature extraction from the image is , in the detector In the branch, assume The target is identified as In this case, the category information of the target label is temporarily changed manually. And calculate the detection loss. New category Compared to the old category The higher the similarity, the greater the loss. The smaller; in the detector In the branch, Perform normal target detection and calculate the detection loss. Let the total classification loss in this training be... ,but: (12) Step 3: Network training.

[0040] Input the training set and validation set into the network built in step 2 for training and validation, respectively. When the loss and accuracy of the validation set converge, select and save the training model for the corresponding training round.

[0041] Table 2 shows the hardware platform parameters used for training in this example.

[0042] Table 2 Hardware Platform Parameters

[0043] Step 4: Network restructuring.

[0044] After the model training is completed in step 3, the structure reparameterization technique is used to seamlessly integrate the expanded convolutional module from step 2 into the original convolutional module, ensuring that the number of model parameters does not increase with the increase of categories. Figure 3 This describes the process of reorganizing modules using structural reparameterization.

[0045] The weights of the convolutional kernels in the fusion single-branch model are , bias is The fused convolution result for: (13) Step 5: Network test.

[0046] The test set is input into the target recognition model after structural reorganization in step 4 for forward inference to obtain the corresponding prediction results, thus obtaining the recognition results for each type of target. Table 3 shows the accuracy of each type of target in the incremental recognition scenario. It can be seen that while the model can recognize new types of targets, it can still maintain the recognition of old types of targets quite well. Figure 5 The recognition results after adding the category confusion loss function show that the model has a good ability to distinguish between the old and new classes.

[0047] Table 3. Accuracy (%) of each target class in incremental recognition scenarios

[0048] Based on the test results in Table 3, the average incremental accuracy and average incremental forgetting rate can be calculated for each incremental process. The results are shown in Table 4. From the three incremental stages, the average incremental accuracy shows a decreasing trend, but the overall performance loss is less than 6%, and the average incremental forgetting rate is also small, with the highest average incremental forgetting rate being less than 10%.

[0049] Table 4. Average incremental accuracy and forgetting rate (%)

Claims

1. A new type of SAR target recognition method based on network model extension, characterized in that, Includes the following steps: Step 1: Dataset Construction; Select SAR images containing the targets to be identified, and label the targets in the images. Crop the images to the size that meets the model input requirements to obtain the labeling results of the corresponding slice images. Before the network training begins, randomly divide all training samples into training set, validation set and test set according to the proportion. Step 2: Target recognition network model construction; The YOLOv7 network model is improved by two parts: expanding the convolutional modules and adding a confusion loss function. At the original kernel size The convolutional module is embedded in parallel with another kernel of size 1. The convolutional module is expanded to have the same number of input and output channels as the original convolutional module, and the parameters of the original convolutional module are used to initialize the new convolutional kernel; when training the model using new class data, the parameters of the original convolutional module are frozen, and only the parameters of the new convolutional module are iterated. The original convolutional module is represented as: （1） in The output feature map, These are the weights of the original convolutional module. For bias; The extended convolutional module is represented as: （2） in To expand the weights of the convolutional module, For bias; Convolution result after convolution module expansion for: （3） In convolution processing with a kernel size of 1 branch, the padding size is adjusted to ensure that the feature map sizes are exactly the same; A class confusion loss function is added to the YOLOv7 loss function to expand the decision boundaries of different classes in the feature space; new classes are included. The feature map obtained by feature extraction from the image is In the detector In the branch, assume The target is identified as In this case, the category information of the target label is temporarily changed manually. And calculate the detection loss. New category Compared to the old category The higher the similarity, the greater the loss. The smaller; in the detector In the branch, Perform normal target detection and calculate the detection loss. Let the total classification loss of this training be... ,but: （4） in These are the weights of the loss function; Step 3: Network training; Input the training set and validation set into the target recognition network model built in step 2 for training and validation, respectively. When the loss and accuracy of the validation set converge, select and save the training model for the corresponding training round. Step 4: Network restructuring; After the model training is completed in step 3, the structure reparameterization technique is used to seamlessly integrate the expanded convolutional module in step 2 into the original convolutional module, so that the number of model parameters does not increase with the increase of categories. The weights of the convolutional kernels in the fusion single-branch model are , bias is The fused convolution result for: （5）。 2. An electronic device, characterized in that, include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to cause the electronic device to perform the method as described in claim 1.

3. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in claim 1.

4. A chip, characterized in that, include: A processor for retrieving and running a computer program from memory, causing a device on which the chip is mounted to perform the method as described in claim 1.

5. A computer program product, characterized in that, The computer program product includes a computer storage medium storing a computer program, the computer program including instructions executable by at least one processor, which, when executed by the at least one processor, implement the method as described in claim 1.

Images