A training method and related apparatus for a SAR image classification model

Abstract

This application discloses a training method and related apparatus for a SAR image classification model, relating to the field of artificial intelligence technology. The method involves weighted fusion and centering of the teacher projector head based on weighting factors to obtain a distillation balanced projector head. The total loss of the iteration is calculated based on the ground truth labels, the distillation balanced projector head, the student projector head, the first projector head, and the second projector head. Since the weighting factors are inversely correlated with the number of images belonging to the image category of the sample SAR images in the training data, the long-tail problem of the training data can be improved. Furthermore, the total loss is obtained through supervised and unsupervised joint training based on the ground truth labels, the distillation balanced projector head, the student projector head, the first projector head, and the second projector head of all sample SAR images in the iterative training data, thereby improving the training effect and ultimately enhancing the classification performance of the SAR image classification model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a training method and related apparatus for a SAR image classification model. Background Technology

[0002] Synthetic Aperture Radar (SAR) is an active microwave remote sensing imaging radar that acquires images by emitting coherent electromagnetic waves to illuminate the earth's surface and then receiving the scattered echoes from targets on the surface. SAR images reflect the scattering characteristics of ground objects and targets to microwaves, and have the unique advantage of all-day, all-weather imaging.

[0003] SAR image classification models are typically used to detect and identify target features and models from SAR images. However, current SAR image classification models used for SAR image category discovery suffer from poor classification performance. Summary of the Invention

[0004] In view of the above problems, this application provides a training method and related apparatus for a SAR image classification model, so as to improve the classification performance of the SAR image classification model. The specific solution is as follows:

[0005] The first aspect of this application provides a training method for a SAR image classification model, including multiple iterations, wherein the iterations include:

[0006] The training data for the iteration is obtained, which includes sample SAR images and ground truth labels, wherein the ground truth labels identify the image category;

[0007] For any of the sample SAR images, obtain the first enhanced image and the second enhanced image of the sample feature image of the sample SAR image;

[0008] The first enhanced image of the sample SAR image is input into the student model and the teacher model to obtain the student projection head output by the student model and the teacher projection head output by the teacher model. The student model is a SAR image classification model to be trained. The SAR image classification model consists of an encoder and a first projector. The teacher model is a mirror model of the SAR image classification model.

[0009] The teacher projection head is weighted, fused and centered based on a weighting factor to obtain a distilled balanced projection head. The weighting factor is inversely correlated with the number of images in the training data that belong to the image category of the sample SAR image.

[0010] The first enhanced image and the second enhanced image are input to the perception projection module to obtain the first projection head corresponding to the first enhanced image and the second projection head corresponding to the second enhanced image output by the perception projection module. The perception projection module shares an encoder with the SAR image classification model.

[0011] The total loss of the iteration is calculated based on the ground truth labels of all the sample SAR images in the training data of the iteration, the distillation balanced projector, the student projector, the first projector, and the second projector.

[0012] If the preset training completion conditions are met, the training ends.

[0013] If the training completion conditions are not met, update the model parameters and execute the next iteration.

[0014] In one possible implementation, the encoder is a frequency attention module, and the first projector is a linear layer;

[0015] The sensing projection module consists of the encoder and the second projector, the second projector being a multilayer sensor.

[0016] In one possible implementation, acquiring a first enhanced image and a second enhanced image of the sample feature image of the sample SAR image includes:

[0017] The sample feature image is obtained by linear projection and location encoding of the sample SAR image;

[0018] The sample feature image is resized back to its original size by random cropping, subjected to random color distortion, and transformed by random Gaussian blur to generate a first enhanced image and a second enhanced image of the sample feature image.

[0019] In one possible implementation, the total loss of the iteration is calculated based on the ground-value labels of all the sample SAR images in the training data of the iteration, the distillation balanced projector, the student projector, the first projector, and the second projector, including:

[0020] The distillation balance projection head and the student projection head are normalized based on the preset normalized softmax function. The normalized result corresponding to the distillation balance projection head is used as the soft pseudo label, and the normalized result corresponding to the student projection head is used as the predicted value.

[0021] The first and second projection heads of the sample SAR image are used as the first and second contrast values, respectively.

[0022] The first projection head corresponding to the first enhanced image of other sample SAR images is obtained as a comparison value. The other sample images are images in the training data of the iteration other than the sample SAR images.

[0023] Based on the soft pseudo-labels, the predicted values, and the ground truth labels, calculate the classification learning loss;

[0024] Based on the comparison value, the first comparison value, and the second comparison value, the representation learning loss is calculated;

[0025] The sum of the classification learning loss and the representation learning loss is calculated as the total loss of the iteration.

[0026] In one possible implementation, the classification learning loss is calculated based on the soft pseudo-label, the predicted value, and the ground truth label, including:

[0027] Based on the soft pseudo-labels and the predicted values, unsupervised classification learning is performed on the iterative training data using the first unsupervised contrastive loss function to obtain the first unsupervised loss value. The first unsupervised contrastive loss function is equal to the cross-entropy function minus the average entropy maximization function.

[0028] Based on the predicted value and the true value label, supervised classification learning is performed on the iterative training data using a first supervised contrastive loss function to obtain a first supervised loss value. The first supervised contrastive loss function is the cross-entropy loss function.

[0029] Based on the preset first joint training weighting coefficient, the first unsupervised loss value and the first supervised loss value are weighted and added together to obtain the classification learning loss.

[0030] In one possible implementation, the representation learning loss is calculated based on the contrast value, the first contrast value, and the second contrast value, including:

[0031] Obtain the direct contrast value from the contrast comparison value, wherein the direct contrast value is the first projection head corresponding to the first enhanced image of the same type of sample image, and the same type of sample image is an image in the training data of the iteration that is of the same image category as the sample SAR image;

[0032] Based on the positive contrast value, the first contrast value, and the second contrast value, unsupervised contrastive learning is performed on the training data of the iteration using the second unsupervised contrastive loss function to obtain the second unsupervised loss value;

[0033] Based on the positive contrast value, the negative contrast value, the first contrast value, and the second contrast value, supervised contrastive learning is performed on the iterative training data using the second supervised contrastive loss function to obtain the second supervised loss value;

[0034] Based on the preset second joint training weighting coefficient, the second unsupervised loss value and the second supervised loss value are weighted and added together to obtain the representation learning loss.

[0035] A second aspect of this application provides a training apparatus for a SAR image classification model, including a joint training module, the joint training module comprising:

[0036] The training data acquisition unit is used to acquire the training data of the iteration, the training data including sample SAR images and ground truth labels, the ground truth labels identifying the image category;

[0037] The image enhancement unit is used to acquire, for any one of the sample SAR images, a first enhanced image and a second enhanced image of the sample feature image of the sample SAR image;

[0038] The first projection unit is used to input the first enhanced image of the sample SAR image into the student model and the teacher model, and to obtain the student projection head output by the student model and the teacher projection head output by the teacher model. The student model is a SAR image classification model to be trained. The SAR image classification model consists of an encoder and a first projector. The teacher model is a mirror model of the SAR image classification model.

[0039] A distillation balancing unit is used to perform weighted fusion and centering processing on the teacher projection head based on a weighting factor to obtain a distillation balanced projection head. The weighting factor is inversely correlated with the number of images in the training data that belong to the image category of the sample SAR image.

[0040] The second projection unit is used to input the first enhanced image and the second enhanced image to the perception projection module, and to obtain the first projection head corresponding to the first enhanced image and the second projection head corresponding to the second enhanced image output by the perception projection module. The perception projection module shares an encoder with the SAR image classification model.

[0041] The total loss calculation unit is used to calculate the total loss of the iteration based on the ground truth labels of all the sample SAR images in the training data of the iteration, the distillation balanced projector, the student projector, the first projector, and the second projector.

[0042] The training condition determination unit is used to end training if the preset training completion condition is met; otherwise, it updates the model parameters and executes the next iteration.

[0043] A third aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the training method for a SAR image classification model according to the first aspect or any implementation thereof.

[0044] A fourth aspect of this application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0045] The memory is used to store computer programs;

[0046] The processor is used to execute the computer program so that the electronic device can implement the training method of the SAR image classification model of the first aspect or any implementation thereof.

[0047] The fifth aspect of this application provides a computer storage medium carrying one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the motion recognition method for web page objects described in the first aspect or any implementation thereof.

[0048] Using the above technical solution, this application provides a training method and related apparatus for a SAR image classification model. It acquires iterative training data, including sample SAR images and ground truth labels, with the ground truth labels identifying image categories. For any sample SAR image, a first enhanced image and a second enhanced image of the sample feature image are acquired. The first enhanced image of the sample SAR image is input into a student model and a teacher model, acquiring the student projector output by the student model and the teacher projector output by the teacher model. The student model is the SAR image classification model to be trained, consisting of an encoder and a first projector. The teacher model is a mirror model of the SAR image classification model. The teacher projector is weighted, fused, and centered based on a weight factor to obtain a distilled balanced projector. The weight factor is inversely correlated with the number of images belonging to the image category of the sample SAR image in the training data. The first enhanced image and the second enhanced image are input into a perceptual projection module, acquiring the first projector corresponding to the first enhanced image and the second projector corresponding to the second enhanced image output by the perceptual projection module. The perceptual projection module shares an encoder with the SAR image classification model. The total loss of the iteration is calculated based on the ground truth labels of all sample SAR images in the iterative training data, the distillation balanced projector, the student projector, the first projector, and the second projector. If the preset training completion condition is met, the training ends. If the training completion condition is not met, the model parameters are updated and the next iteration is executed. This scheme performs weighted fusion and centering processing on the teacher projector based on the weight factor to obtain the distillation balanced projector, and calculates the total loss of the iteration based on the ground truth labels, the distillation balanced projector, the student projector, the first projector, and the second projector. Since the weight factor is inversely correlated with the number of images belonging to the image category of the sample SAR images in the training data, it can improve the long-tail problem of the training data. Furthermore, the total loss is obtained by supervised and unsupervised joint training based on the ground truth labels of all sample SAR images in the iterative training data, the distillation balanced projector, the student projector, the first projector, and the second projector, thereby improving the training effect and thus improving the classification effect of the SAR image classification model. Attached Figure Description

[0049] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0050] Figure 1 A schematic diagram of a system architecture is provided for this application;

[0051] Figure 2 A schematic diagram of an optional hardware structure for terminal 100 is shown;

[0052] Figure 3 A schematic diagram of the structure of a server 200 is shown;

[0053] Figure 4 A flowchart illustrating a training method for a SAR image classification model provided in an embodiment of this application;

[0054] Figure 5 A flowchart illustrating the specific implementation of a SAR image classification method provided in this application embodiment;

[0055] Figure 6a This is a schematic diagram illustrating the specific structure of a SAR image classification model provided in an embodiment of this application.

[0056] Figure 6b This is a schematic diagram of the specific structure of a frequency module provided in an embodiment of this application;

[0057] Figure 6c This is a schematic diagram of the specific structure of an attention module provided in an embodiment of this application;

[0058] Figure 7 This is a schematic diagram of the specific structure of a BKD-CL model provided in an embodiment of this application;

[0059] Figure 8a This example demonstrates a basic framework for SimCLR;

[0060] Figure 8b This is a schematic diagram of the specific structure of a multilayer sensor provided in an embodiment of this application;

[0061] Figure 9 A flowchart illustrating a training method for a SAR image classification model provided in an embodiment of this application;

[0062] Figure 10 A flowchart illustrating a total loss calculation method provided in an embodiment of this application;

[0063] Figure 11 A schematic diagram of the structure of a training device for a SAR image classification model provided in an embodiment of this application;

[0064] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0065] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0066] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0067] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0068] This application can be applied to the field of artificial intelligence technology, specifically to the field of synthetic aperture radar automatic identification, for image classification tasks that detect and identify target features and models from SAR images.

[0069] Currently, the technical challenges are: (1) Traditional SAR ATR relies on a large amount of well-annotated training data, but data annotation is difficult in real-world scenarios. When mobile military targets in SAR images undergo technological upgrades, such as adding towers to tanks or replacing aircraft chassis and wing structures, these modifications often lead to the emergence of new target categories. (2) The distribution of real-world data samples is unbalanced, often exhibiting a long-tail distribution, meaning that a few categories occupy a large number of samples. This unbalanced data distribution can easily lead to the model overfitting to the head category while ignoring the tail category, thereby reducing the model's generalization ability. Therefore, to promote the intelligent implementation of SAR systems, it is necessary to solve the problems of new category identification and unknown data distribution.

[0070] To address the aforementioned technical problems, this invention proposes a long-tailed SAR category discovery method based on the BKD-CL model. This method involves jointly training the SAR image classification model using both supervised and unsupervised methods to overcome the poor generalization ability of the SAR image classification model caused by imbalanced sample distribution. It enables category discovery in open environments with unknown data distributions, and improves the system's intelligence level through SAR's autonomous learning and the identification of new targets.

[0071] This application can be applied, but is not limited to, to applications with model training capabilities or cloud services provided by cloud-side servers, which will be described in detail below:

[0072] See Figure 1 , Figure 1A schematic diagram of a system architecture is shown. The system may include a terminal 100 and a server 200. The server 200 may include one or more servers (…). Figure 1 (The example includes a server), and the server 200 can provide the method provided in the embodiments of this application to one or more terminals.

[0073] The terminal 100 may have a training application for a SAR image classification model installed on it. The application and webpage can provide an interface. The terminal 100 can receive relevant parameters input by the user on the training interface of the SAR image classification model and send the parameters to the server 200. The server 200 can obtain the processing result based on the received parameters and return the processing result to the terminal 100.

[0074] It should be understood that in some optional implementations, the terminal 100 can also complete the action of obtaining the processing result based on the received parameters on its own, without the need for the server to cooperate. This application embodiment is not limited to this.

[0075] The following description Figure 1 The product form of the mid-terminal 100;

[0076] The terminal 100 in this application embodiment can be a mobile phone, tablet computer, wearable device, vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc., and this application embodiment does not impose any restrictions on it.

[0077] Figure 2 A schematic diagram of an optional hardware structure for terminal 100 is shown.

[0078] refer to Figure 2 As shown, the terminal 100 may include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160 (optional), a speaker 161 (optional), a microphone 162 (optional), a headphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190, and other components. Those skilled in the art will understand that... Figure 2 These are merely examples of terminals or multi-functional devices and do not constitute a limitation on terminals or multi-functional devices. They may include more or fewer components than shown in the illustration, or combine certain components, or use different components.

[0079] The input unit 130 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the portable multi-functional device. Specifically, the input unit 130 may include a touchscreen 131 (optional) and / or other input devices 132. The touchscreen 131 can collect touch operations performed by the user on or near it (such as operations performed by the user using fingers, knuckles, styluses, or any suitable object on or near the touchscreen), and drive the corresponding connection devices according to a pre-set program. The touchscreen can detect the user's touch actions, convert the touch actions into touch signals and send them to the processor 170, and can receive and execute commands sent by the processor 170; the touch signal includes at least touch point coordinate information. The touchscreen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, various types of touchscreens, such as resistive, capacitive, infrared, and surface acoustic wave, can be used to implement the touchscreen. Besides the touchscreen 131, the input unit 130 may also include other input devices. Specifically, other input devices 132 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0080] Among them, the input device 132 can receive input data, etc.

[0081] The display unit 140 can be used to display information input by the user or information provided to the user, various menus of the terminal 100, interactive interfaces, file display, and / or playback of any multimedia file. In this embodiment, the display unit 140 can be used to display the training interface of the SAR image classification model, processing results, etc.

[0082] The memory 120 can be used to store instructions and data. The memory 120 may primarily include an instruction storage area and a data storage area. The data storage area can store various types of data, such as multimedia files and text. The instruction storage area can store software units such as operating systems, applications, and instructions required for at least one function, or subsets or extended sets thereof. It may also include non-volatile random access memory. It provides the processor 170 with hardware, software, and data resources for managing the computing device, supporting control software and applications. It is also used for storing multimedia files, as well as storing running programs and applications.

[0083] The processor 170 is the control center of the terminal 100. It connects various parts of the terminal 100 via various interfaces and lines. By running or executing instructions stored in the memory 120 and calling data stored in the memory 120, it performs various functions and processes data of the terminal 100, thereby controlling the terminal device as a whole. Optionally, the processor 170 may include one or more processing units; preferably, the processor 170 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 170. In some embodiments, the processor and memory can be implemented on a single chip; in some embodiments, they can also be implemented separately on independent chips. The processor 170 can also be used to generate corresponding operation control signals, send them to the corresponding components of the computing processing device, read and process data in the software, especially read and process data and programs in the memory 120, so that the various functional modules therein perform corresponding functions, thereby controlling the corresponding components to act according to the instructions.

[0084] The memory 120 can be used to store software code related to the training method of the SAR image classification model, and the processor 170 can execute the steps of the training method of the SAR image classification model, and can also schedule other units (such as the above-mentioned input unit 130 and display unit 140) to achieve the corresponding functions.

[0085] The radio frequency unit 110 (optional) can be used for receiving and transmitting signals during information transmission or calls. For example, it can receive downlink information from the base station and process it for the processor 170; additionally, it can transmit uplink data to the base station. Typically, the RF circuit includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, the radio frequency unit 110 can also communicate wirelessly with network devices and other devices. This wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0086] In this embodiment of the application, the radio frequency unit 110 can send data to the server 200 and receive the processing results sent by the server 200.

[0087] It should be understood that the radio frequency unit 110 is optional and can be replaced with other communication interfaces, such as a network port.

[0088] The terminal 100 also includes a power supply 190 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the processor 170 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0089] Terminal 100 also includes an external interface 180, which can be a standard Micro USB interface or a multi-pin connector, which can be used to connect terminal 100 to other devices for communication or to connect a charger to charge terminal 100.

[0090] Although not shown, terminal 100 may also include a flash, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with various functions, etc., which will not be described in detail here. Some or all of the methods described below can be applied to, for example... Figure 2 In the terminal 100 shown.

[0091] The following description Figure 1 The product form of the mid-range server 200;

[0092] Figure 3 A schematic diagram of the structure of a server 200 is shown, as follows: Figure 3 As shown, server 200 includes bus 201, processor 202, communication interface 203, and memory 204. Processor 202, memory 204, and communication interface 203 communicate with each other via bus 201.

[0093] Bus 201 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0094] The processor 202 can be any one or more of the following processors: central processing unit (CPU), graphics processing unit (GPU), microprocessor (MP), or digital signal processor (DSP).

[0095] Memory 204 may include volatile memory, such as random access memory (RAM). Memory 204 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0096] The memory 204 can be used to store software code related to the training method of the SAR image classification model, and the processor 202 can execute the steps of the training method of the chip's SAR image classification model, and can also schedule other units to achieve corresponding functions.

[0097] It should be understood that the aforementioned terminal 100 and server 200 can be centralized or distributed devices. The processors (e.g., processor 170 and processor 202) in the aforementioned terminal 100 and server 200 can be hardware circuits (such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), general-purpose processors, digital signal processors (DSPs), microprocessors or microcontrollers, etc.) or combinations of these hardware circuits. For example, the processor can be a hardware system with instruction execution capabilities, such as a CPU or DSP, or a hardware system without instruction execution capabilities, such as an ASIC or FPGA, or a combination of the aforementioned hardware systems without instruction execution capabilities and hardware systems with instruction execution capabilities.

[0098] This application provides a method for training a SAR image classification model. The training method for the SAR image classification model according to this application will be described in detail below with reference to the accompanying drawings.

[0099] Reference Figure 4 , Figure 4 A flowchart illustrating a training method for a SAR image classification model provided in this application embodiment is shown below. Figure 4 As shown in the embodiment of this application, a training method for a SAR image classification model includes multiple iterations. Each iteration may include steps S401 to S404, which are described in detail below.

[0100] S401. Obtain the training data for the iteration.

[0101] In this embodiment, the training data includes sample SAR images and ground truth labels, with the ground truth labels identifying the image category.

[0102] S402. For any sample SAR image, obtain the first enhanced image and the second enhanced image of the sample feature image of the sample SAR image.

[0103] S403. Input the first enhanced image of the sample SAR image into the student model and the teacher model, and obtain the student projection head output by the student model and the teacher projection head output by the teacher model.

[0104] In this embodiment, the student model is a SAR image classification model to be trained, which consists of an encoder and a first projector, and the teacher model is a mirror image of the SAR image classification model.

[0105] In one alternative embodiment, the encoder is a frequency attention module and the first projector is a linear layer.

[0106] S404. Based on the weighting factor, the teacher projection head is weighted, fused, and centered to obtain a distilled balanced projection head.

[0107] In this embodiment, the weighting factor is inversely correlated with the number of images in the training data that belong to the image category of the sample SAR images.

[0108] S405. Input the first enhanced image and the second enhanced image into the perception projection module, and obtain the first projection head corresponding to the first enhanced image and the second projection head corresponding to the second enhanced image output by the perception projection module.

[0109] In this embodiment, the perception projection module consists of an encoder and a second projector. The second projector is a multilayer perceptron, and the perception projection module shares the encoder with the SAR image classification model. That is, the perception projection module is a FAVIT module.

[0110] S406. Calculate the total loss of the iteration based on the ground truth labels of all sample SAR images in the iterative training data, the distillation balanced projector, the student projector, the first projector, and the second projector.

[0111] S407. If the preset training completion conditions are met, the training ends.

[0112] S408. If the training completion condition is not met, update the model parameters and execute the next iteration.

[0113] As can be seen from the above technical solutions, the training method for a SAR image classification model provided in this application performs weighted fusion and centering processing on the teacher projector based on weight factors to obtain a distillation balanced projector. The total loss of the iteration is calculated based on the ground truth label, the distillation balanced projector, the student projector, the first projector, and the second projector. Since the weight factor is inversely correlated with the number of images belonging to the image category of the sample SAR image in the training data, it can improve the long-tail problem of the training data. Furthermore, the total loss is obtained by supervising and unsupervised joint training based on the ground truth label, the distillation balanced projector, the student projector, the first projector, and the second projector of all sample SAR images in the iterative training data, thereby improving the training effect and thus improving the classification effect of the SAR image classification model.

[0114] See Figure 5 , Figure 5 A flowchart illustrating the specific implementation of a SAR image classification method provided in this application embodiment is shown below. Figure 5 As shown, this method specifically includes S501 to S507, as follows:

[0115] S501. Construct a SAR image classification model.

[0116] In this embodiment, the SAR image classification model consists of a frequency attention module and linear layers. The SAR image classification model takes a SAR image to be identified as input, extracts image features from the SAR image through N backbone networks, and outputs the category identification result based on the image features through linear layers (i.e., fully connected layers). The linear layers are normalized linear layers, and the number of neurons in the output layer equals the number of labeled categories. The frequency attention module includes N backbone networks, and each backbone network includes multiple frequency modules and multiple attention modules.

[0117] Figure 6a This is a schematic diagram illustrating the specific structure of a SAR image classification model provided in an embodiment of this application.

[0118] like Figure 6a As shown, the frequency attention module includes N backbone networks, and each backbone network includes 2 frequency modules and 4 attention modules.

[0119] Figure 6b This is a schematic diagram of the specific structure of a frequency module provided in an embodiment of this application. Figure 6b As shown, the frequency module includes a normalized Norm layer, a spectrum-gated network, and a multilayer perceptron (MLP). The spectrum-gated network comprises a Fast Fourier Transform (FFT) layer, a weighted gating layer, and an Inverse Fourier Transform (IFFT) layer. The frequency module uses the spectrum-gated network to introduce a global filter, thereby capturing different frequency components of the SAR image to understand local frequencies.

[0120] First, regarding the input The frequency module obtains the result through 2D Fast Fourier Transform (FFT). Spectrum This allows the physical space to be converted into the spectrum space, as shown in formula (1):

[0121] (1);

[0122] in, This represents the 2D Fast Fourier Transform. It is a complex tensor, representing The spectrum.

[0123] Furthermore, the frequency module uses learnable weight parameters through a weighted gating layer. Determine the weight of each frequency component In order to properly capture the lines and edges of the SAR image and obtain the modulated spectrum. The formula is as follows (2):

[0124] (2);

[0125] in, Represents element-wise multiplication, global filter With spectrum They have the same dimension.

[0126] Furthermore, the frequency module uses IFFT to modulate the spectrum. Transform back to physical space to achieve The updated formula is as follows (3):

[0127] (3);

[0128] in, This represents the inverse of the 2D Fast Fourier Transform.

[0129] Furthermore, the frequency module performs nonlinear transformation and processing on the output data of the spectrum gating network through MLP to enhance the expressive power of the frequency module.

[0130] Figure 6c This is a schematic diagram illustrating the specific structure of an attention module provided in an embodiment of this application. Figure 6c As shown, the attention module includes a normalization layer, a multi-head self-attention layer, and a multilayer perceptron (MLP).

[0131] First, the attention module generates initial values ​​for q, k, and v using a fully connected layer through a multi-head self-attention layer. Then, it adjusts these values ​​using reshaping and dimension manipulation. Finally, it uses slicing operations to obtain individual Q, K, and V values. The attention module introduces a multi-head self-attention mechanism to enhance the network's stability and robustness.

[0132] Furthermore, the attention module uses MLP to perform nonlinear transformation and processing on the output data of the multi-head self-attention layer to enhance the expressive power of the attention module.

[0133] S502. Construct a joint training model for training SAR image classification models.

[0134] In this embodiment, the jointly trained model is called the BKD-CL model. Figure 7 This is a schematic diagram of the structure of a joint training model provided in an embodiment of this application, as shown below. Figure 7 As shown, the joint training model includes a preprocessing module, a data augmentation module, a SAR image classification model to be trained as a student model, a teacher model of the SAR image classification model, a multilayer perceptron, and a loss function module.

[0135] The preprocessing module comprises a linear projection layer and a location coding layer connected in sequence. The input data for the data preprocessing module is training data, which includes both labeled and unlabeled data. The preprocessing module outputs sample feature images of the sample SAR images.

[0136] The SAR image classification model to be trained includes a frequency attention module and a linear layer, which are used to output the student classification results. The teacher model and the student model have completely identical structures and can be different versions of the SAR image classification model to be trained, used to output the teacher classification results. For specific structures, please refer to the above embodiment.

[0137] In this embodiment, the FAVIT module includes a frequency attention module in the SAR image classification model and a multilayer perceptron connected to the output of the frequency attention module. The FAVIT module is built based on SimCLR (Simple Framework for Contrastive Learning of Representations, self-supervised learning based on contrastive learning). The frequency attention module of the BKD-CL model constitutes the encoder in the basic framework of SimCLR, and the multilayer perceptron constitutes the projection head in the basic framework of SimCLR. Figure 8 illustrates a basic framework of SimCLR, which includes:

[0138] The data augmentation module is used to resize the sample SAR image x back to its original size by random cropping, and to transform the input sample SAR image by random color distortion and random Gaussian blurring, generating two augmented images of the same sample SAR image x. and , and Form a pair of positive samples.

[0139] encoder This is used to extract representation vectors from various enhanced images, for example, by using a residual network to extract representation vectors to obtain representation vectors. and ,in, and These are all representation vectors output after the average pooling layer.

[0140] Projector This is used to map representation vectors to the latent space of contrastive learning to obtain projection results. For example, it can be extracted using an MLP (Multilayer Perceptron) with hidden layers. and In practical applications, the specific mechanisms of the encoder and projector (also known as the projection head) can be selected according to requirements.

[0141] The contrastive loss function is used to extract 2N augmented images from a training subset consisting of N sample SAR images (i.e., the training dataset used in one iteration), where each sample SAR image generates 2 augmented images. For a sample SAR image x in the training subset, when a positive sample pair of sample SAR image x is determined ( , After that, enhance the image. Together with 2(N-1) enhanced images, they form negative sample pairs, denoted as ( , ), For 2N enhanced images, excluding positive sample pairs ( , Enhanced images beyond ) The quantity is 2(N-1).

[0142] The contrastive loss function is used to perform comparative learning based on the projection results generated by positive and negative sample pairs, so as to maximize the consistency of the projection results of positive sample pairs. For example, let... Comparison loss function Defined as formula (4):

[0143] (4);

[0144] in, It is an indicator function that evaluates to 1. This represents the temperature parameter.

[0145] In this embodiment, the category discovery task is a joint task of supervised and unsupervised classification. The contrastive learning in the SimCLR model can learn the image features of unlabeled sample SAR images by training the model to identify which data points are similar or different.

[0146] In this embodiment, the frequency attention module outputs a first representation vector and a second representation vector based on the input first enhanced image and second enhanced image, respectively. The multilayer perceptron is used to output a first projection head and a second projection head based on the first representation vector and the second representation vector, respectively.

[0147] Figure 8b This is a schematic diagram of a specific structure of a multilayer sensor provided in an embodiment of this application, as shown below. Figure 8b As shown, the multilayer perceptron includes 2048 hidden layers and 256 output layers. Therefore, the tensor changes from (64, 384) to a projected feature of (64, 256) after passing through the multilayer perceptron.

[0148] The loss function module includes a distillation balancing module, a softmax module, and a total loss calculation module. The distillation balancing module is connected to the output of the linear layer of the teacher model. The softmax module is connected to the output of both the distillation balancing module and the SAR image classification model. The total loss calculation module is connected to the output of both the multilayer perceptron and the softmax module. The distillation balancing module comprises a sequentially connected fusion module and a centering module, used to receive the teacher projection head output from the teacher model and output a fused teacher projection head. The softmax module uses a normalized softmax function to output soft pseudo-labels based on the received teacher projection head and predicted values ​​based on the received student projection head output from the student model.

[0149] The total loss calculation module calculates the total loss based on the received ground truth label, soft pseudo label, predicted value, and comparison value, and updates the model parameters based on the total loss. The comparison value includes a first projection head as the first comparison value, a second projection head as the second comparison value, and a reference comparison value. The reference comparison value is determined based on the first projection head of multiple negative samples of the sample SAR image.

[0150] S503. Acquire sample SAR image data and divide the sample SAR image data into labeled data and unlabeled data.

[0151] In this embodiment, the sample SAR image data includes various types of sample SAR images. The sample SAR images are acquired by a high-resolution spotlight synthetic aperture radar sensor under various scalable conditions such as target obstruction, camouflage, and configuration changes. Optionally, the radar operates in the X-band and uses HH polarization.

[0152] Specifically, the number of categories of sample SAR images in the sample SAR image data is n. n1 categories are randomly selected as known categories, and the remaining n2 categories are as unknown categories, where n = n1 + n2.

[0153] In each known class of sample SAR images, a preset first proportion of sample SAR images are selected as the first class of images. These first class images are then used as labeled images to construct labeled data. .in This represents the u-th labeled image. Let represent the truth label of the u-th labeled image, where the truth label of a labeled image is the category, and u∈[1, 2, 3]. ], This indicates the number of labeled images.

[0154] From the known class of sample SAR images, those other than the first class images are designated as the second class images. Using the second class images and the unknown class sample SAR images as unlabeled images, unlabeled data is constructed. ,in, This represents the v-th unlabeled image. This represents the label of the v-th unlabeled image. The label of an unlabeled image can be null, and v∈[1, 2, 3]. ], This indicates the number of unlabeled images.

[0155] Taking a sample SAR image data set comprising 10 categories as an example, five categories are selected as known categories and five categories as unknown categories. From the sample SAR images of the known categories, 50% of the sample SAR images are selected as labeled images to construct labeled data. The remaining 50% of the sample SAR images of the known class and the sample SAR images of the unknown class are used as unlabeled images to constitute unlabeled data. .

[0156] S504. Using the training data, perform supervised and unsupervised joint training on the SAR image classification model based on the BKD-CL model until the preset training completion conditions are met, and obtain the trained SAR image classification model.

[0157] In this embodiment, the joint supervised and unsupervised training of the SAR image classification model based on the BKD-CL model includes multiple iterative training sessions. The execution method for each iterative training session is described in the above embodiment.

[0158] In this embodiment, the training data includes labeled data. and unlabeled data That is, the training data includes + 32 SAR images were used as the training data. The training data was input into the BKD-CL model, with a maximum of 300 iterations. The batch size of the training data for each iteration was 32, meaning 32 SAR images were used to fit the BKD-CL model each time. It should be noted that the dimensions of the training data in a single iteration are (32, 3, 224, 224), where 32 is the batch size, 3 is the number of channels, and 224*224 is the width and height of the sample SAR image.

[0159] In this embodiment, the initial learning rate is 0.001, and the SGD (Stochastic Gradient Descent) optimization method is used to update all parameters of the SAR image classification model. The training completion conditions include the error being less than a preset error threshold or the maximum number of iterations being met. That is, iteration stops when the error is less than the preset error threshold or the maximum number of iterations is met, and the trained SAR image classification model is obtained.

[0160] Specifically, the model training platform consists of two NVIDIA GeForce RTX 2080Ti GPUs, and Table 1 lists the training parameter configurations.

[0161] Table 1

[0162]

[0163] S505. Input the validation data into the trained SAR image classification model, and output the predicted category of the SAR image samples belonging to the known class and the predicted category of the SAR image samples belonging to the unknown class in the validation data through the SAR image classification model.

[0164] In this embodiment, unlabeled data Input the data into the trained SAR image classification model, and output the predicted category corresponding to each sample SAR image in the unlabeled data.

[0165] In this embodiment, accuracy is a measure of the overall correctness of the model. The ratio of correctly identified sample SAR images to the validation data is obtained by counting the number of correctly predicted categories in SAR images.

[0166] In this embodiment, the evaluation metrics include a first classification accuracy, a second classification accuracy, and a third classification accuracy. The first classification accuracy represents the ratio of correctly identified sample SAR images to the verification data; the second classification accuracy represents the ratio of correctly identified sample SAR images of known classes to all sample SAR images of known classes; and the third classification accuracy represents the ratio of correctly identified sample SAR images of unknown classes to all sample SAR images of unknown classes.

[0167] Specifically, the accuracy of the model is calculated based on various evaluation indicators, as shown in formula (5):

[0168] (5);

[0169] Where TP represents the number of samples with the same predicted category and label that are correct, and N represents the total number of samples.

[0170] Furthermore, to address the category discovery problem caused by imbalanced data distribution, this scheme introduces an imbalance factor. To change the number of sample SAR images of known classes in the unlabeled data, that is, the number of images of the second class. Number of SAR images of unknown class This allows for testing under different long-tail ratios, i.e. .

[0171] Table 2 lists the classification accuracy performance of the BKD-CL model provided in this solution and existing technologies such as GCD (Generalized Category Discovery), SimGCD (Simple Generalized Category Discovery), and ORCA (Open World with Uncertainty Based Adaptive Margin) models at different long-tail ratios. Table 2 shows that the BKD-CL model exhibits higher accuracy in classifying known classes, unknown classes, and all classes at different long-tail ratios. As the long-tail ratio increases, the third-class classification accuracy of the other three models decreases significantly. In contrast, the balanced knowledge distillation of the BKD-CL model effectively addresses the problem of low third-class classification accuracy.

[0172] Table 2

[0173]

[0174] S506. If the preset verification conditions are met, output the trained SAR image classification model.

[0175] In this embodiment, the verification conditions include that the accuracy of the first classification, the accuracy of the second classification, and the accuracy of the third classification are all not less than the corresponding index thresholds, and the model accuracy is not less than the preset accuracy threshold.

[0176] S507. Obtain the SAR image to be identified, input the SAR image to be identified into the trained SAR image classification model, and obtain the category identification result of the SAR image to be identified output by the SAR image classification model.

[0177] This application embodiment implements a training method for SAR image classification models through joint training models. Figure 9 A flowchart illustrating a training method for a SAR image classification model provided in this application embodiment is shown below. Figure 9 As shown, the training method includes multiple iterations. Optionally, the batch size of the training data in a single iteration is a preset batch size. The m-th iteration includes:

[0178] S901. The sample feature image is obtained by linearly projecting and position encoding the sample SAR image in the training data of the m-th iteration through the preprocessing module.

[0179] In this embodiment, taking the i-th sample SAR image as an example, the sample SAR image in the training data is input to the preprocessing module. After patching the sample SAR image through the linear projection layer of the preprocessing module, the tensor of the resulting image block is (64, 196, 384). The position encoding layer embeds the position code into the image block and normalizes it, resulting in a normalized image block tensor of (64, 384). Here, the position code expresses the position information of the image block in the sample SAR image. The sample feature image xi output by the position encoding module is composed of a normalized image block with a tensor of (64, 384).

[0180] S902. Input the sample feature image into the data augmentation module, and generate a first augmented image and a second augmented image of the sample feature image through the data augmentation module.

[0181] In this embodiment, the data augmentation module adjusts the size of the sample feature image back to its original size by random cropping, random color distortion, and random Gaussian blur transformation of the input sample feature image, generating two enhancements of the same sample feature image, namely the first enhancement image and the second enhancement image.

[0182] like Figure 7 As shown, the sample feature image The data is input into the data augmentation module, which generates the first augmented image of the sample feature image. Second enhanced image .

[0183] S903. Input the first enhanced image into the student model, output the representation vector of the first enhanced image to the linear layer through the frequency attention module, and output the student projection head through the linear layer.

[0184] like Figure 7 As shown, the first enhanced image The input is fed into the student model (i.e., the classification model of the SAR image to be trained), and processed through the frequency attention module. Encode and output the first representation vector ,Will The input is fed into a linear layer, and then processed by the linear layer. Projecting, outputting student projector head .

[0185] S904. Input the first enhanced image into the teacher model, output the representation vector of the first enhanced image to the linear layer through the frequency attention module, and output the teacher projection head through the linear layer.

[0186] In this embodiment, the teacher model is a mirror image of the SAR image classification model. Specifically, the teacher model and the student model can be different versions of the SAR image classification model.

[0187] like Figure 7 As shown, The input is fed into the teacher model, which performs the same operations as the student model and outputs the teacher projection head. .

[0188] In another alternative embodiment, the first projection head output by the student model is directly copied to obtain the teacher projection head.

[0189] S905. Input the teacher projection head into the distillation balancing module in the loss function module. The distillation balancing module performs weighted fusion of the teacher projection head based on the weight factor through the fusion module. The centering module performs centering processing on the weighted fused teacher projection head and outputs the distillation balanced projection head.

[0190] like Figure 7 As shown, the fusion module in the equilibrium distillation module is based on a weighting factor. For teacher projection head To perform weighted fusion, specifically, to Divide by the temperature parameter of the teacher model Later introduction right Weighted fusion is performed, and the teacher output is updated after centralization through a centralized module to form a distillation-balanced projection head. .

[0191] In self-supervised learning, when a model maps multiple input data to the same feature representation, considering only a subset of the data representations while ignoring features from other data samples can significantly negatively impact the model's robustness. Using a centralized module can prevent model collapse and avoid a single feature dominating the model's representation. Specifically, this is achieved by adding a bias term. In the teacher model, The update strategy uses the exponential moving average method.

[0192] S906. Input the distillation balance projector and the student projector into the Softmax module. The Softmax module outputs soft pseudo-labels based on the distillation balance projector and outputs predicted values ​​based on the student projector.

[0193] In this embodiment, the student output is divided by the temperature parameter of the student model. Then, the softmax function is applied to update the student output to obtain the predicted value.

[0194] like Figure 7As shown, the Softmax module is used to... Normalization is performed to obtain the predicted value. ,right Normalization is performed to obtain soft pseudo-labels. .

[0195] S907. Input the first enhanced image and the second enhanced image into the FAVIT module, output the representation vectors of the first enhanced image and the second enhanced image to the multilayer perceptron through the frequency attention module, and output the first projection head and the second projection head through the multilayer perceptron.

[0196] In this embodiment, the first enhanced image and the second enhanced image are input to the frequency attention module to obtain the representation vector of the first enhanced image and the representation vector of the second enhanced image output by the frequency attention module as the encoder, that is, the first representation vector and the second representation vector. The first representation vector and the second representation vector are input to the multilayer perceptron to obtain the projection head of the first representation vector and the projection head of the second representation vector output by the multilayer perceptron as the projection head, that is, the first projection head and the second projection head.

[0197] like Figure 7 As shown, and The input is fed into the frequency attention module, which outputs the first representation vector. Second representation vector ,Will and The input is sent to the multilayer perceptron, and the multilayer perceptron outputs the first projection head. Second projection head .

[0198] S908. Input the true value label, soft false label, predicted value, and comparison value into the total loss calculation module.

[0199] In this embodiment, the comparison values ​​are the first comparison value and the second comparison value, which are also the first projection value and the second projection value, such as... Figure 7 As shown, the truth label Soft fake labels Predicted values and the first comparison value Second comparison value Input into the total loss calculation module.

[0200] S909. For the training data of the m-th iteration, the total loss is calculated by substituting the ground truth label, soft pseudo label, predicted value, first comparison value, second comparison value, and reference comparison value of each sample SAR image into the total loss function through the total loss calculation module.

[0201] In this embodiment, the first and second projection heads of the sample SAR image are used as the first and second contrast values, respectively. The first projection heads corresponding to the first augmented images of other sample SAR images are obtained and used as the control contrast values. These other sample images are images other than the sample SAR images in the iterative training data. The classification learning loss is calculated based on the soft pseudo-labels, predicted values, and ground truth labels. The representation learning loss is calculated based on the control contrast values, the first contrast value, and the second contrast value. The sum of the classification learning loss and the representation learning loss is calculated as the total loss for the iteration.

[0202] It should be noted that for a specific method of calculating the total loss, please refer to [link / reference needed]. Figure 10 .

[0203] S910. Determine whether the preset training completion conditions have been met.

[0204] In this embodiment, the training completion conditions include the number of iterations reaching a threshold, or the total loss being less than a preset loss threshold.

[0205] S911. If yes, end the iteration and configure the SAR image classification model based on the current values ​​of the model parameters to obtain the trained SAR image classification model.

[0206] S912. If not, update the model parameters and return to S901 to execute the (m+1)th iteration.

[0207] Figure 10 A schematic diagram of a total loss calculation block diagram provided in an embodiment of this application, as shown below. Figure 10 As shown, S909 calculates the total loss for the m-th iteration by substituting the predicted value, contrast value, soft pseudo-label, and ground truth label of each sample SAR image in the training data of one iteration into the total loss calculation module.

[0208] In this embodiment, the total loss calculation module performs joint learning of representation learning and classification learning based on the total loss function. Representation learning includes supervised and unsupervised learning, and classification learning also includes supervised and unsupervised learning. Specifically, the representation learning in the total loss calculation module includes processing labeled data from the frequency attention module in the student branch. This enables supervised learning and self-supervised learning on training data. The classification learning in the total loss calculation module comes from the cross-entropy loss between predicted values ​​and soft pseudo-labels, or the cross-entropy loss between predicted values ​​and ground truth labels.

[0209] In this embodiment, the total loss calculation module calculates the total loss by substituting the ground truth label, soft pseudo label, predicted value, first comparison value, second comparison value, and control comparison value of each sample SAR image into the total loss function. The specific method includes:

[0210] S1. Based on the soft pseudo-labels and predicted values, the first unsupervised contrastive loss function is used to perform unsupervised classification learning on the iterative training data to obtain the first unsupervised loss value.

[0211] In this embodiment, the first unsupervised contrastive loss function is equal to the cross-entropy function minus the average entropy maximization function.

[0212] In this embodiment, self-supervised training can use soft pseudo-labels to transfer a small amount of labeled sample SAR image data to a large amount of unlabeled sample SAR image data, thereby improving the quality of features. This scheme uses self-distillation technology for self-supervised training classification learning, thereby solving the problem of new class discovery.

[0213] Self-supervised classification learning is performed on the training data using the cross-entropy loss function between the soft pseudo-labels and the predicted values. The loss function of self-supervised classification learning is equal to the cross-entropy minus the result of maximizing the average entropy, as shown in formula (6):

[0214] (6);

[0215] in, Indicates a soft fake tag. This represents the predicted value. and These are the average prediction and entropy of the batch (iterative training data), respectively. ( The result of maximizing the average entropy is calculated using formulas (7) and (8), respectively:

[0216] (7);

[0217] (8).

[0218] In this embodiment, directly connecting the linear layer to the frequency attention module can improve the quality of soft pseudo-labels, thereby achieving good new category classification performance. However, in the category discovery problem of long-tail data, the trained model tends to favor the head class. During distillation, the prediction information of the tail class may be overwhelmed by the prediction information of the head class. Therefore, a teacher model guided by such a biased model may perform worse.

[0219] Based on this, this scheme introduces a balanced knowledge distillation method through a distillation balance module, which transfers knowledge by considering prior information about the number of categories. Specifically, the weighting factor of the sample SAR image is calculated as shown in formula (9):

[0220] (9);

[0221] in, It's a hyperparameter. It is the sum of the number of times the sample SAR image belongs to the corresponding category in the entire training data.

[0222] Therefore, by introducing the weight factor update formula (6) for the sample SAR image, the first unsupervised contrast loss function is obtained. As in formula (10):

[0223] (10).

[0224] S2. Based on the predicted values ​​and true labels, supervised classification learning is performed on the iterative training data using the first supervised contrastive loss function to obtain the first supervised loss value.

[0225] In this embodiment, the first supervised comparison loss function is the cross-entropy loss function.

[0226] like Figure 10 As shown, using predicted values and truth labels The cross-entropy loss between the training data is used for supervised classification learning. The first supervised contrastive loss function is shown in formula (11):

[0227] (11);

[0228] in, This indicates that there is labeled data. Is with The corresponding truth label.

[0229] S3. Based on the preset first joint training weighting coefficient, the first unsupervised loss value and the first supervised loss value are weighted and added together to obtain the classification learning loss.

[0230] Therefore, through the first joint training weighting coefficients combination and Obtain the classification learning loss function For example, in formula (12):

[0231] (12).

[0232] S4. Based on the control comparison value, the direct comparison value, the first comparison value, and the second comparison value, the second unsupervised comparison loss function is used to perform unsupervised comparison learning on the iterative training data to obtain the second unsupervised loss value.

[0233] In this embodiment, taking the training data of this iteration including N sample images as an example, for Comparison value Including other sample images The first projection head corresponding to the first enhanced image is directly facing the contrast value. Including similar sample images The first projection head corresponding to the first enhanced image is directly facing the contrast value and extracts it from the control contrast value.

[0234] In this embodiment, during representation learning, the features of labeled SAR images of the same category are brought closer together in the feature space, while the features of SAR images of different categories are pushed apart.

[0235] Specifically, the second unsupervised contrastive loss function As in formula (13):

[0236] (13).

[0237] in, It is the index corresponding to a batch of images (the training data of the m-th iteration), for example, =N. This means including all of the same batch except for The index of the set of first augmented images of other sample images besides the sample images. The first projection head is the first enhanced image corresponding to other sample images.

[0238] In unsupervised contrastive learning tasks, the ground truth label is empty, meaning there is no annotation information. Positive sample pairs are typically two augmented images of the same SAR image, i.e., two enhanced images after image augmentation. Negative samples come from other samples in the same batch. Therefore, unsupervised learning solves the classification problem of multiple samples belonging to different categories. However, when there are multiple SAR images of the same category, only the augmented positive samples can be classified into the same category, while other samples belonging to the same category are far removed from that category. Therefore, supervised learning is introduced.

[0239] S5. Based on the positive contrast value, the first contrast value, and the second contrast value, supervised contrastive learning is performed on the iterative training data using the second supervised contrastive loss function to obtain the second supervised loss value.

[0240] In this embodiment, the second supervised contrastive loss function As in formula (14):

[0241] (14);

[0242] in, For the same batch and The index of all other sample images with the same truth label (denoted as class sample images), It is the first projection head corresponding to the first enhanced image of the same type of sample images, directly opposite the contrast value. These are pre-configured temperature parameters.

[0243] S6. Based on the preset second joint training weighting coefficient, the second unsupervised loss value and the second supervised loss value are weighted and added together to obtain the representation learning loss.

[0244] In this embodiment, the second joint training weighting coefficient is used. Combinatorial supervised contrastive loss function Comparison loss function with unsupervised Obtain the loss function for representation learning As in formula (15):

[0245] (15).

[0246] In this embodiment, the FAVIT module used for contrastive learning includes a frequency attention module with learnable features and a multilayer perceptron. By directly cascading the multilayer perceptron with the frequency attention module, the projection features of the image can be obtained. Traditional self-supervised classification networks are built on projection features, and such networks are more suitable for proxy tasks than classification tasks.

[0247] S7. Calculate the sum of the classification learning loss and the representation learning loss as the total loss for the iteration.

[0248] In this embodiment, the total loss function of the total loss calculation module As in formula (16):

[0249] (16).

[0250] In summary, this step calculates the loss value for this iteration by substituting the predicted value, contrast value, soft pseudo-label, and ground truth label of each sample SAR image in the training data of this iteration into the total loss calculation module.

[0251] As can be seen from the above technical solutions, this solution introduces the FAVIT module to enhance data representation by using supervised and unsupervised contrastive learning on labeled and training data. Cascading a projection head with entropy regularization can improve the recognition accuracy of new classes. To address the long-tail distribution problem in open worlds, introducing balanced knowledge distillation can improve the recognition accuracy of tail classes.

[0252] The above describes a training method for a SAR image classification model provided by the embodiments of this application. The following will describe the apparatus for performing the above-described training method for a SAR image classification model.

[0253] Please see Figure 11 , Figure 11 This is a schematic diagram of a training device for a SAR image classification model provided in an embodiment of this application. The training device for the SAR image classification model is configured within the client, such as... Figure 11 As shown, the training device 1100 for the SAR image classification model includes a joint training module for performing multiple iterations, the joint training module comprising:

[0254] The training data acquisition unit 1101 is used to acquire the training data of the iteration, the training data including sample SAR images and ground truth labels, the ground truth labels identifying the image category;

[0255] Image enhancement unit 1102 is used to acquire, for any one of the sample SAR images, a first enhanced image and a second enhanced image of the sample feature image of the sample SAR image;

[0256] The first projection unit 1103 is used to input the first enhanced image of the sample SAR image into the student model and the teacher model, and obtain the student projection head output by the student model and the teacher projection head output by the teacher model. The student model is a SAR image classification model to be trained. The SAR image classification model consists of an encoder and a first projector. The teacher model is a mirror model of the SAR image classification model.

[0257] Distillation balancing unit 1104 is used to perform weighted fusion and centering processing on the teacher projection head based on weighting factors to obtain a distillation balanced projection head. The weighting factors are inversely correlated with the number of images in the training data that belong to the image category of the sample SAR image.

[0258] The second projection unit 1105 is used to input the first enhanced image and the second enhanced image to the perception projection module, and obtain the first projection head corresponding to the first enhanced image and the second projection head corresponding to the second enhanced image output by the perception projection module. The perception projection module shares an encoder with the SAR image classification model.

[0259] The total loss calculation unit 1106 is used to calculate the total loss of the iteration based on the ground truth labels of all the sample SAR images in the training data of the iteration, the distillation balanced projector, the student projector, the first projector, and the second projector.

[0260] The training condition determination unit 1107 is used to end training if the preset training completion condition is met; if the training completion condition is not met, the model parameters are updated and the next iteration is executed.

[0261] In one possible implementation, the encoder is a frequency attention module, and the first projector is a linear layer;

[0262] The sensing projection module consists of the encoder and the second projector, the second projector being a multilayer sensor.

[0263] In one possible implementation, when the image enhancement unit acquires the first and second enhanced images of the sample feature images of the sample SAR image, it specifically performs the following functions:

[0264] The sample feature image is obtained by linear projection and location encoding of the sample SAR image;

[0265] The sample feature image is resized back to its original size by random cropping, subjected to random color distortion, and transformed by random Gaussian blur to generate a first enhanced image and a second enhanced image of the sample feature image.

[0266] In one possible implementation, the total loss calculation unit, when calculating the total loss of the iteration based on the ground truth labels of all the sample SAR images in the training data of the iteration, the distillation balanced projector, the student projector, the first projector, and the second projector, is specifically used for:

[0267] The distillation balance projection head and the student projection head are normalized based on the preset normalized softmax function. The normalized result corresponding to the distillation balance projection head is used as the soft pseudo label, and the normalized result corresponding to the student projection head is used as the predicted value.

[0268] The first and second projection heads of the sample SAR image are used as the first and second contrast values, respectively.

[0269] The first projection head corresponding to the first enhanced image of other sample SAR images is obtained as a comparison value. The other sample images are images in the training data of the iteration other than the sample SAR images.

[0270] Based on the soft pseudo-labels, the predicted values, and the ground truth labels, calculate the classification learning loss;

[0271] Based on the comparison value, the first comparison value, and the second comparison value, the representation learning loss is calculated;

[0272] The sum of the classification learning loss and the representation learning loss is calculated as the total loss of the iteration.

[0273] In one possible implementation, the total loss calculation unit, when calculating the classification learning loss based on the soft pseudo-labels, the predicted values, and the ground truth labels, specifically performs the following:

[0274] Based on the soft pseudo-labels and the predicted values, unsupervised classification learning is performed on the iterative training data using the first unsupervised contrastive loss function to obtain the first unsupervised loss value. The first unsupervised contrastive loss function is equal to the cross-entropy function minus the average entropy maximization function.

[0275] Based on the predicted value and the true value label, supervised classification learning is performed on the iterative training data using a first supervised contrastive loss function to obtain a first supervised loss value. The first supervised contrastive loss function is the cross-entropy loss function.

[0276] Based on the preset first joint training weighting coefficient, the first unsupervised loss value and the first supervised loss value are weighted and added together to obtain the classification learning loss.

[0277] In one possible implementation, the total loss calculation unit, when calculating the representation learning loss based on the contrast value, the first contrast value, and the second contrast value, specifically performs the following:

[0278] Obtain the direct contrast value from the contrast comparison value, wherein the direct contrast value is the first projection head corresponding to the first enhanced image of the same type of sample image, and the same type of sample image is an image in the training data of the iteration that is of the same image category as the sample SAR image;

[0279] Based on the positive contrast value, the first contrast value, and the second contrast value, unsupervised contrastive learning is performed on the training data of the iteration using the second unsupervised contrastive loss function to obtain the second unsupervised loss value;

[0280] Based on the positive contrast value, the negative contrast value, the first contrast value, and the second contrast value, supervised contrastive learning is performed on the iterative training data using the second supervised contrastive loss function to obtain the second supervised loss value;

[0281] Based on the preset second joint training weighting coefficient, the second unsupervised loss value and the second supervised loss value are weighted and added together to obtain the representation learning loss.

[0282] This application also provides an electronic device in its embodiments. (See reference...) Figure 12 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 12 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0283] like Figure 9 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage device 1208 into a random access memory (RAM) 1203. When the electronic device is powered on, the RAM 1203 also stores various programs and data required for the operation of the electronic device. The processing unit 1201, ROM 1202, and RAM 1203 are interconnected via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0284] Typically, the following devices can be connected to I / O interface 1205: input devices 1206 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 1207 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1208 including, for example, memory cards, hard drives, etc.; and communication devices 1209. Communication device 1209 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 12 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have instead.

[0285] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the SAR image classification model training methods provided in this application.

[0286] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the SAR image classification model training methods provided in this application.

[0287] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0288] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0289] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0290] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A training method for a SAR image classification model, characterized in that, This includes multiple iterations, wherein the iterations include: The training data for the iteration is obtained, which includes sample SAR images and ground truth labels, wherein the ground truth labels identify the image category; For any of the sample SAR images, obtain the first enhanced image and the second enhanced image of the sample feature image of the sample SAR image; The first enhanced image of the sample SAR image is input into the student model and the teacher model to obtain the student projection head output by the student model and the teacher projection head output by the teacher model. The student model is a SAR image classification model to be trained. The SAR image classification model consists of an encoder and a first projector. The teacher model is a mirror model of the SAR image classification model. The teacher projection head is weighted, fused and centered based on a weighting factor to obtain a distilled balanced projection head. The weighting factor is inversely correlated with the number of images in the training data that belong to the image category of the sample SAR image. The first enhanced image and the second enhanced image are input to the perception projection module to obtain the first projection head corresponding to the first enhanced image and the second projection head corresponding to the second enhanced image output by the perception projection module. The perception projection module shares an encoder with the SAR image classification model. The total loss of the iteration is calculated based on the ground truth labels of all the sample SAR images in the training data of the iteration, the distillation balanced projector, the student projector, the first projector, and the second projector. If the preset training completion conditions are met, the training ends. If the training completion conditions are not met, update the model parameters and execute the next iteration.

2. The training method for the SAR image classification model according to claim 1, characterized in that, The encoder is a frequency attention module, and the first projector is a linear layer; The sensing projection module consists of the encoder and the second projector, the second projector being a multilayer sensor.

3. The training method for the SAR image classification model according to claim 1, characterized in that, The first and second enhanced images of the sample feature images of the sample SAR image obtained include: The sample feature image is obtained by linear projection and location encoding of the sample SAR image; The sample feature image is resized back to its original size by random cropping, subjected to random color distortion, and transformed by random Gaussian blur to generate a first enhanced image and a second enhanced image of the sample feature image.

4. The training method for the SAR image classification model according to claim 3, characterized in that, The total loss of the iteration is calculated using the ground truth labels of all sample SAR images in the training data based on the iteration, the distillation balanced projector, the student projector, the first projector, and the second projector, including: The distillation balance projection head and the student projection head are normalized based on the preset normalized softmax function. The normalized result corresponding to the distillation balance projection head is used as the soft pseudo label, and the normalized result corresponding to the student projection head is used as the predicted value. The first and second projection heads of the sample SAR image are used as the first and second contrast values, respectively. The first projection head corresponding to the first enhanced image of other sample SAR images is obtained as a comparison value. The other sample SAR images are images in the training data of the iteration other than the sample SAR images. Based on the soft pseudo-label, the predicted value, and the ground truth label, calculate the classification learning loss; Based on the comparison value, the first comparison value, and the second comparison value, the representation learning loss is calculated; The sum of the classification learning loss and the representation learning loss is calculated as the total loss of the iteration.

5. The training method for the SAR image classification model according to claim 4, characterized in that, The step of calculating the classification learning loss based on the soft pseudo-label, the predicted value, and the ground truth label includes: Based on the soft pseudo-labels and the predicted values, unsupervised classification learning is performed on the iterative training data using the first unsupervised contrastive loss function to obtain the first unsupervised loss value. The first unsupervised contrastive loss function is equal to the cross-entropy function minus the average entropy maximization function. Based on the predicted value and the true value label, supervised classification learning is performed on the iterative training data using a first supervised contrastive loss function to obtain a first supervised loss value. The first supervised contrastive loss function is the cross-entropy loss function. Based on the preset first joint training weighting coefficient, the first unsupervised loss value and the first supervised loss value are weighted and added together to obtain the classification learning loss.

6. The training method for the SAR image classification model according to claim 4, characterized in that, The step of calculating the representation learning loss based on the contrast value, the first contrast value, and the second contrast value includes: Obtain the direct contrast value from the contrast comparison value, wherein the direct contrast value is the first projection head corresponding to the first enhanced image of the same type of sample image, and the same type of sample image is an image in the training data of the iteration that is of the same image category as the sample SAR image; Based on the positive contrast value, the first contrast value, and the second contrast value, unsupervised contrastive learning is performed on the training data of the iteration using the second unsupervised contrastive loss function to obtain the second unsupervised loss value; Based on the positive contrast value, the negative contrast value, the first contrast value, and the second contrast value, supervised contrastive learning is performed on the iterative training data using the second supervised contrastive loss function to obtain the second supervised loss value; Based on the preset second joint training weighting coefficient, the second unsupervised loss value and the second supervised loss value are weighted and added together to obtain the representation learning loss.

7. A training device for a SAR image classification model, characterized in that, Includes a joint training module, which includes: The training data acquisition unit is used to acquire iterative training data, which includes sample SAR images and ground truth labels, wherein the ground truth labels identify the image category. An image enhancement unit is used to acquire, for any one of the sample SAR images, a first enhanced image and a second enhanced image of the sample feature image of the sample SAR image; The first projection unit is used to input the first enhanced image of the sample SAR image into the student model and the teacher model, and to obtain the student projection head output by the student model and the teacher projection head output by the teacher model. The student model is a SAR image classification model to be trained. The SAR image classification model consists of an encoder and a first projector. The teacher model is a mirror model of the SAR image classification model. A distillation balancing unit is used to perform weighted fusion and centering processing on the teacher projection head based on a weighting factor to obtain a distillation balanced projection head. The weighting factor is inversely correlated with the number of images in the training data that belong to the image category of the sample SAR image. The second projection unit is used to input the first enhanced image and the second enhanced image to the perception projection module, and to obtain the first projection head corresponding to the first enhanced image and the second projection head corresponding to the second enhanced image output by the perception projection module. The perception projection module shares an encoder with the SAR image classification model. The total loss calculation unit is used to calculate the total loss of the iteration based on the ground truth labels of all the sample SAR images in the training data of the iteration, the distillation balanced projector, the student projector, the first projector, and the second projector. The training condition determination unit is used to end training if the preset training completion condition is met; otherwise, it updates the model parameters and executes the next iteration.

8. A computer program product, characterized in that, It includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the training method for the SAR image classification model as described in any one of claims 1 to 6.

9. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the training method of the SAR image classification model as described in any one of claims 1 to 6.

10. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the training method for the SAR image classification model as described in any one of claims 1 to 6.

Images