ISAR image recognition method and system based on CLIP and continual learning

Abstract

The application provides an ISAR image recognition method and system based on CLIP and continuous learning, comprising the following steps: step 1, ISAR image preprocessing and generating corresponding explanatory text; step 2, using a CLIP model to extract image features of the ISAR image and text features of the explanatory text; step 3, constructing a continuous learning cache model and performing parameter fine-tuning; and step 4, inputting the ISAR image for recognition inference and outputting the recognition result with the highest matching degree according to the input ISAR image. The application uses the powerful cross-modal capability of the CLIP model to link the ISAR image and the explanatory text, and uses the continuous learning method to inhibit the possible catastrophic forgetting in the fine-tuning process, so that the recognition method and system can process more abundant information, and solve the problem that the current ISAR image recognition system has limited processing capacity when facing ISAR images with abundant information.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and ISAR image recognition technology, specifically to an ISAR image recognition method and system based on CLIP and continuous learning. Background Technology

[0002] ISAR (Independent SAR) is a technology capable of high-resolution imaging of non-cooperative targets, possessing ultra-long-range imaging capabilities and operating in various weather conditions and complex environments. ISAR images, generated by processing radar echo signals, contain rich information such as the target's position, distance, shape, relative attitude, and motion state. In recent years, deep learning-based image classification methods have achieved great success and have been widely applied in many fields such as remote sensing, medicine, and security. The deep learning-based ISAR image recognition methods derived from this have also proven to have powerful performance; well-trained deep neural networks can identify a large number of ISAR images with extremely high speed and accuracy. To address the difficulty of acquiring ISAR images, techniques for few-shot recognition have also been developed. However, most current methods only train neural networks based on ISAR images and label information.

[0003] Patent document CN116994143A (application number: CN202311044597.5) discloses a maneuvering spatial target recognition method based on complex region maps (Transformers), which incorporates a contrastive learning method based on a self-supervised learning strategy to enhance the deep neural network's processing capability for ISAR images. While this type of method reduces reliance on label information by further utilizing the features of the image itself, it still doesn't fully utilize prior knowledge and cue information beyond the ISAR image. Therefore, a method is needed that can receive more input information to improve the deep neural network's ISAR image recognition capability. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the purpose of this invention is to provide an ISAR image recognition method and system based on CLIP and continuous learning.

[0005] The ISAR image recognition method based on CLIP and continuous learning provided by the present invention includes:

[0006] Step 1: Perform ISAR image preprocessing to generate corresponding explanatory text;

[0007] Step 2: Use the CLIP model to extract image features from the ISAR image and text features from the descriptive text;

[0008] Step 3: Build a continuous learning caching model and fine-tune its parameters;

[0009] Step 4: Input the ISAR image for recognition and reasoning, and output the recognition result with the highest matching degree based on the input ISAR image.

[0010] Preferably, step 1 includes:

[0011] Receive ISAR image data and perform data augmentation, including: rotation, flipping, cropping, scaling, color transformation, noise injection, and blurring.

[0012] Create corresponding explanatory text, which includes the ISAR image category and data augmentation method;

[0013] Obtain the ISAR image dataset {x i} N，C With the explanatory text dataset {t i} N，C , where N represents the image x in the dataset. i Or text t i The total number, where C represents the number of categories.

[0014] Preferably, step 2 includes:

[0015] Image encoder f using CLIP model image Extracting ISAR image x i Image features f image (x i ;θ), a text encoder f using the CLIP model text Extract the corresponding description text t i Text features f text (t i ;θ), where θ represents the parameters of the CLIP model; for the CLIP model, f image (x i ;θ) and f text (t i ;θ) are both 512-dimensional vectors;

[0016] Input multiple sets of ISAR images and explanatory text to obtain multiple pairs of image features and text features;

[0017] Feature extraction is performed on all images and text in the training dataset;

[0018] Using the extracted image and text features, a similarity score is calculated between the image and text features:

[0019] s a，b，θ =CosSim<f image (x a ;θ), f text (tb ;θ)>

[0020] Where: CosSim<*, *> represents the calculation of cosine similarity; when a = b, s a，a，θ Represents ISAR image x a Its corresponding explanatory text t a The similarity score of the features; when a≠b, s a，b，θ Represents ISAR image x a With ISAR image x b The corresponding explanatory text t b The similarity score of the features.

[0021] Preferably, step 3 includes:

[0022] Randomly select k images from each category, resulting in a total of C·k images, where C represents the number of categories. Use the image features and ground truth labels of these images to create one-hot encodings of keys and values, respectively, to construct an initial cache model for continuous learning. This cache model contains a total of C·k key-value pairs, where the key portion, composed of the image feature vectors, is represented as... The value part, composed of one-hot encoding of the label truth value, is represented as: During training, the key part F of the cache model is fine-tuned. Cache This allows for a better reflection of the image features of the dataset;

[0023] During training, for each input ISAR image x i Image features are used to calculate their corresponding descriptive text t for each category. c The similarity score of the features yields a C-dimensional vector f. CLIP That is, the classifier of the CLIP model itself for image x i The output;

[0024] Calculated image features f image (x i ;θ) and the similarity score of each image feature in the cache model, i.e.: Through nonlinear transformation Mapping it from [-1, 1] to [0, 1], β is a hyperparameter, and then with L Cache Multiplication yields a C-dimensional vector f. Cache That is, continuously learning the caching model for image x i The output;

[0025] The entire model applies to image x i The output is represented as f Out =f CLIP +αf Cache , where α is a hyperparameter;

[0026] Use the output f Out With image x i The true class labels are used to construct the cross-entropy loss, and gradient descent is used to optimize the key parameters of the continuously learning cached model.

[0027] Preferably, step 4 includes: for each ISAR image to be identified x test The final output f is obtained by extracting features through the image encoder of the CLIP model, calculating the output of the classifier of the CLIP model itself, and calculating the output of the fine-tuned cache model. Out The ISAR image x is obtained after applying the softmax function. test The recognition results.

[0028] The ISAR image recognition system based on CLIP and continuous learning provided by the present invention includes:

[0029] Module M1: Performs ISAR image preprocessing and generates corresponding explanatory text;

[0030] Module M2: Uses the CLIP model to extract image features from ISAR images and text features from descriptive text.

[0031] Module M3: Builds a continuous learning caching model and fine-tunes its parameters;

[0032] Module M4: Inputs an ISAR image for recognition and inference, and outputs the recognition result with the highest matching degree based on the input ISAR image.

[0033] Preferably, the module M1 includes:

[0034] Receive ISAR image data and perform data augmentation, including: rotation, flipping, cropping, scaling, color transformation, noise injection, and blurring.

[0035] Create corresponding explanatory text, which includes the ISAR image category and data augmentation method;

[0036] Obtain the ISAR image dataset {x i} N，C With the explanatory text dataset {t i} N，C , where N represents the image x in the dataset i Or text t i The total number, where C represents the number of categories.

[0037] Preferably, the module M2 includes:

[0038] Image encoder f using CLIP model image Extracting ISAR image x iImage features f image (x i ;θ), a text encoder f using the CLIP model text Extract the corresponding description text t i Text features f text (t i ;θ), where θ represents the parameters of the CLIP model; for the CLIP model, f image (x i ;θ) and f text (t i ;θ) are both 512-dimensional vectors;

[0039] Input multiple sets of ISAR images and explanatory text to obtain multiple pairs of image features and text features;

[0040] Feature extraction is performed on all images and text in the training dataset;

[0041] Using the extracted image and text features, a similarity score is calculated between the image and text features:

[0042] s a，b，θ =CosSim<f image (x a ;θ), f text (t b ;θ)>

[0043] Where: CosSim<*, *> represents the calculation of cosine similarity; when a = b, s a，a，θ Represents ISAR image x a Its corresponding explanatory text t a The similarity score of the features; when a≠b, s a，b，θ Represents ISAR image x a With ISAR image x b The corresponding explanatory text t b The similarity score of the features.

[0044] Preferably, the module M3 includes:

[0045] Randomly select k images from each category, resulting in a total of C·k images, where C represents the number of categories. Use the image features and ground truth labels of these images to create one-hot encodings of keys and values, respectively, to construct an initial cache model for continuous learning. This cache model contains a total of C·k key-value pairs, where the key portion, composed of the image feature vectors, is represented as... The value part, composed of one-hot encoding of the label truth value, is represented as: During training, the key part F of the cache model is fine-tuned. Cache This allows for a better reflection of the image features of the dataset;

[0046] During training, for each input ISAR image x i Image features are used to calculate their corresponding descriptive text t for each category. c The similarity score of the features yields a C-dimensional vector f. CLIP That is, the classifier of the CLIP model itself for image x i The output;

[0047] Calculated image features f image (x i ;θ) and the similarity score of each image feature in the cache model, i.e.: Through nonlinear transformation Mapping it from [-1, 1] to [0, 1], β is a hyperparameter, and then with L Cache Multiplication yields a C-dimensional vector f. Cache That is, continuously learning the caching model for image x i The output;

[0048] The entire model applies to image x i The output is represented as f Out =f CLIP +αf Cache , where α is a hyperparameter;

[0049] Use the output f Out With image x i The true class labels are used to construct the cross-entropy loss, and gradient descent is used to optimize the key parameters of the continuously learning cached model.

[0050] Preferably, the module M4 includes: for each ISAR image to be identified x test The final output f is obtained by extracting features through the image encoder of the CLIP model, calculating the output of the classifier of the CLIP model itself, and calculating the output of the fine-tuned cache model. Out The ISAR image x is obtained after applying the softmax function. test The recognition results.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] (1) This invention utilizes the powerful cross-modal capability of the CLIP model to link ISAR images and explanatory text, enabling ISAR image recognition methods and systems to process richer information across modalities;

[0053] (2) The present invention uses a continuous learning method to suppress catastrophic forgetting that may occur during the fine-tuning process, and improves the recognition accuracy by randomly selecting images of each category to construct a cache model and fine-tuning the cache model parameters.

[0054] (3) In the training process, the present invention only needs to fine-tune the parameters of the key part in the cache model. The number of parameters to be adjusted is: C·k·512, where C is the total number of ISAR image categories to be identified, and k is a setting related to the cache size configuration. Typical values ​​are: 4, 8, 16. The common range of parameter quantity is: 0.02M to 1M. Therefore, the training resources are consumed very little. Attached Figure Description

[0055] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0056] Figure 1 A flowchart of an ISAR image recognition method and system based on CLIP and continuous learning;

[0057] Figure 2 Figure 1 shows a method for extracting image features and descriptive text features from an ISAR image recognition method and system based on CLIP and continuous learning.

[0058] Figure 3 This diagram illustrates the construction of a continuous learning cache model for an ISAR image recognition method and system based on CLIP and continuous learning.

[0059] Figure 4 This is an example diagram illustrating ISAR image recognition using a finely tuned model, based on CLIP and continuous learning, as part of an ISAR image recognition method and system. Detailed Implementation

[0060] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0061] Example 1

[0062] This invention provides an ISAR image recognition method based on CLIP and continuous learning, comprising the following steps: Step 1: ISAR image preprocessing and generating corresponding explanatory text; Step 2: Using the CLIP model to extract image features of the ISAR image and text features of the explanatory text; Step 3: Constructing a continuous learning cache model and fine-tuning its parameters; Step 4: Inputting the ISAR image for recognition inference, and outputting the recognition result with the highest matching degree based on the input ISAR image.

[0063] Step 1 includes: receiving ISAR image data and performing appropriate data augmentation, including rotation, flipping, cropping, scaling, color transformation, noise injection, and blurring. Creating corresponding descriptive text, which may include the ISAR image category, data augmentation method, and other descriptive information (such as shape, distance, motion state, etc.). This yields the ISAR image dataset {x}. i} N，C With the explanatory text dataset {t i} N，C , where N represents the total number of images or text in the dataset, and C represents the number of categories.

[0064] Step 2 includes: using an image encoder f with a CLIP model. image Extracting ISAR image x i Image features f image (x i ;θ), a text encoder f using the CLIP model text Extract the corresponding description text t i Text features f text (t i ;θ), where θ represents the parameters of the CLIP model. For the CLIP model, f image (x i ;θ) and f text (t i Both θ and φ are 512-dimensional vectors. By inputting multiple sets of ISAR images and explanatory text, multiple pairs of image and text features can be obtained. Feature extraction is performed on all images and text in the training dataset.

[0065] Using the extracted image and text features, a similarity score can be calculated between the image and text features:

[0066] s a，b，θ =CosSim<f image (x a ;θ), f text (t b ;θ)>

[0067] Where: CosSim<*, *> represents the calculation of cosine similarity; when a = b, s a,a,θ Represents ISAR image x a Its corresponding explanatory text t a The similarity score of the features; when a≠b, s a，b，θ Represents ISAR image x a With ISAR image x b The corresponding explanatory text t b The similarity score of the features.

[0068] Step 3 includes: randomly selecting k images from each category, resulting in a total of C·k images. These images are then used to construct an initial cache model for continuous learning, using one-hot encoding of their image features and ground truth labels as keys and values, respectively. This cache model contains a total of C·k key-value pairs. Specifically, the key portion, composed of image feature vectors, is represented as... The value part, composed of one-hot encoding of the label truth value, is represented as: During training, our goal is to fine-tune the key part F of the cache model. Cache This allows it to better reflect the image features of the dataset.

[0069] During training, for each input ISAR image x i Image features: Calculate the corresponding descriptive text t for each category c The similarity score of the features can be used to obtain a C-dimensional vector f. CLIP This is the CLIP model's own classifier for image x. i The output; the calculated image features f image (x i ;θ) and the similarity score of each image feature in the cache model, i.e.: Through nonlinear transformation Mapping it from [-1, 1] to [0, 1], β is a hyperparameter, and then with L Cache Multiplication yields a C-dimensional vector f. Cache This is a continuous learning cache model for image x i The output.

[0070] The entire model applies to image x i The output is represented as f Out =f CLIP +αf Cache , where α is a hyperparameter. Use the output f Out With image x i The true class labels are used to construct the cross-entropy loss, and gradient descent is used to optimize the key parameters of the continuously learning cached model.

[0071] Step 4 includes: for each ISAR image to be identified x test The final output f is obtained by extracting features through the image encoder of the CLIP model, calculating the output of the classifier of the CLIP model itself, and calculating the output of the fine-tuned cache model. Out The ISAR image x is obtained after applying the softmax function. test The recognition results.

[0072] Example 2

[0073] The present invention also provides an ISAR image recognition system based on CLIP and continuous learning. The ISAR image recognition system based on CLIP and continuous learning can be implemented by executing the process steps of the ISAR image recognition method based on CLIP and continuous learning. That is, those skilled in the art can understand the ISAR image recognition method based on CLIP and continuous learning as a preferred embodiment of the ISAR image recognition system based on CLIP and continuous learning.

[0074] This invention provides an ISAR image recognition system based on CLIP and continuous learning, such as... Figure 1 As shown, it includes:

[0075] Module M1: ISAR image preprocessing and generation of corresponding explanatory text.

[0076] Receive ISAR image data and perform appropriate data augmentation, including rotation, flipping, cropping, scaling, color transformation, noise injection, and blurring. Create corresponding descriptive text, which may include the ISAR image category, data augmentation method, and other descriptive information (e.g., shape, distance, motion state). Obtain the ISAR image dataset {x}. i} N，C With the explanatory text dataset {t i} N，C , where N represents the total number of images or text in the dataset, and C represents the number of categories.

[0077] Module M2: Uses the CLIP model to extract image features from ISAR images and text features from descriptive text.

[0078] like Figure 2 As shown, specifically, module M2 adopts:

[0079] Image encoder f using CLIP model image Extracting ISAR image x i Image features f image (x i ;θ), a text encoder f using the CLIP model text Extract the corresponding description text t i Text features f text (t i ;θ), where θ represents the parameters of the CLIP model. For the CLIP model, f image (x i ;θ) and f text (t iBoth θ and φ are 512-dimensional vectors. By inputting multiple sets of ISAR images and explanatory text, multiple pairs of image and text features can be obtained. Feature extraction is performed on all images and text in the training dataset.

[0080] Using the extracted image and text features, a similarity score can be calculated between the image and text features:

[0081] s a，b，θ =CosSim<f image (x a ;θ), f text (tb;θ)>

[0082] Where: CosSim<*, *> represents the calculation of cosine similarity; when a = b, s a，a，θ Represents ISAR image x a Its corresponding explanatory text t a The similarity score of the features; when a≠b, s a，b，θ Represents ISAR image x a With ISAR image x b The corresponding explanatory text t b The similarity score of the features.

[0083] Module M3: Builds a continuous learning cache model and fine-tunes its parameters.

[0084] like Figure 3 As shown, specifically, module M3 adopts:

[0085] K images are randomly selected from each category, resulting in a total of C·k images. The image features and ground truth labels of these images are one-hot encoded as keys and values, respectively, to construct an initial cache model for continuous learning. This cache model contains a total of C·k key-value pairs. Specifically, the key portion, composed of image feature vectors, is represented as... The value part, composed of one-hot encoding of the label truth value, is represented as: During training, our goal is to fine-tune the key part F of the cache model. Cache This allows it to better reflect the image features of the dataset.

[0086] During training, for each input ISAR image x i Image features:

[0087] Calculate its corresponding descriptive text t for each category. c The similarity score of the features can be used to obtain a C-dimensional vector f. CLIP This is the CLIP model's own classifier for image x. i The output;

[0088] Calculated image features f image (x i ;θ) and the similarity score of each image feature in the cache model, i.e.: Through nonlinear transformation Mapping it from [-1, 1] to [0, 1], β is a hyperparameter, and then with L Cache Multiplication yields a C-dimensional vector f. Cache This is a continuous learning cache model for image x i The output;

[0089] The entire model applies to image x i The output is represented as f Out =f CLIP +αf Cache , where α is a hyperparameter.

[0090] Use the output f Out With image x i The true class labels are used to construct the cross-entropy loss, and gradient descent is applied to optimize the key parameters of the continuously learning cached model. The Adam optimizer is used with a learning rate of 0.0001, and cosine annealing is used to adjust the learning rate. After repeating for 50 epochs, the parameter optimization essentially converges.

[0091] Module M4: Performs recognition inference on the input ISAR image and outputs the recognition result with the highest matching degree based on the input ISAR image. For each ISAR image x to be recognized... test The final output f is obtained by extracting features through the image encoder of the CLIP model, calculating the output of the classifier of the CLIP model itself, and calculating the output of the fine-tuned cache model. Out The ISAR image x is obtained after applying the softmax function. test The recognition result. The recognition process is as follows: Figure 4 As shown.

[0092] In this implementation, the number of parameters required for fine-tuning the training of this invention is C·k·512, where C is the total number of ISAR image categories to be identified, and k is a setting related to the cache size configuration, typically taking values ​​such as 4, 8, or 16. The common range for the number of parameters is 0.02M to 1M.

[0093] In summary, this invention provides an ISAR image recognition method and system based on CLIP and continuous learning. The method preprocesses ISAR images and generates corresponding explanatory text; uses the CLIP model to extract image features from the ISAR image and text features from the explanatory text; constructs a continuous learning cache model and fine-tunes its parameters; performs recognition inference on the input ISAR image, and outputs the recognition result with the highest matching degree based on the input ISAR image. This invention leverages the powerful cross-modal capability of the CLIP model to link ISAR images and explanatory text, while using continuous learning to suppress catastrophic forgetting that may occur during the fine-tuning process. This enables the recognition method and system to process richer information, solving the problem of limited processing capacity of current ISAR image recognition systems when faced with information-rich ISAR images.

[0094] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0095] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. An ISAR image recognition method based on CLIP and continuous learning, characterized in that, include: Step 1: Perform ISAR image preprocessing to generate corresponding explanatory text; Step 2: Use the CLIP model to extract image features from the ISAR image and text features from the descriptive text; Step 3: Build a continuous learning caching model and fine-tune its parameters; Step 4: Input the ISAR image for recognition and reasoning, and output the recognition result with the highest matching degree based on the input ISAR image; Step 2 includes: Image encoder using CLIP model Extracting ISAR images Image features Text encoder using CLIP model Extract the corresponding description text Text features ,in This represents the parameters of the CLIP model; for the CLIP model, and They are all 512-dimensional vectors; Input multiple sets of ISAR images and explanatory text to obtain multiple pairs of image features and text features; Feature extraction is performed on all images and text in the training dataset; Using the extracted image and text features, a similarity score is calculated between the image and text features: in: This indicates the calculation of cosine similarity; when hour, Represents ISAR image Its corresponding explanatory text The similarity score of the features; when hour, Represents ISAR image With ISAR images Corresponding explanatory text The similarity score of the features; Step 3 includes: Randomly selected from each category A total of [number] images were obtained. Zhang Image Representing the number of categories, these images are one-hot encoded with their image features and ground truth labels as keys and values, respectively, to construct an initial cache model for continuous learning. This cache model contains a total of Each key-value pair is represented by a key portion consisting of image feature vectors. The value part, composed of the one-hot encoding of the label truth value, is represented as: During training, the key part of the cache model is fine-tuned. This allows for a better reflection of the image features of the dataset; During training, for each input ISAR image Image features are used to calculate their corresponding descriptive text for each category. The similarity score of the features yields a... dimensional vector That is, the CLIP model's own classifier for images The output; Calculated image features The similarity score between the image features and the cache model, i.e.: Through nonlinear transformation Take it from Mapped to , It is a hyperparameter, and then with Multiplying them together gives one dimensional vector That is, continuous learning of the caching model for images The output; The entire model for images The output is represented as ,in It's a hyperparameter; Use this output With images The true class labels are used to construct the cross-entropy loss, and gradient descent is used to optimize the key parameters of the continuously learning cached model.

2. The ISAR image recognition method based on CLIP and continuous learning according to claim 1, characterized in that, Step 1 includes: Receive ISAR image data and perform data augmentation, including: rotation, flipping, cropping, scaling, color transformation, noise injection, and blurring. Create corresponding explanatory text, which includes the ISAR image category and data augmentation method; Obtain ISAR image dataset With explanatory text dataset ,in Represents images in the dataset or text Total quantity Indicates the number of categories.

3. The ISAR image recognition method based on CLIP and continuous learning according to claim 1, characterized in that, Step 4 includes: for each ISAR image to be identified The final output is obtained by extracting features through the image encoder of the CLIP model, calculating the output of the classifier of the CLIP model itself, and calculating the output of the fine-tuned cache model. ,go through Function to obtain ISAR image The recognition results.

4. An ISAR image recognition system based on CLIP and continuous learning, characterized in that, include: Module M1: Performs ISAR image preprocessing and generates corresponding explanatory text; Module M2: Uses the CLIP model to extract image features from ISAR images and text features from descriptive text. Module M3: Builds a continuous learning caching model and fine-tunes its parameters; Module M4: Inputs an ISAR image for recognition and reasoning, and outputs the recognition result with the highest matching degree based on the input ISAR image; The module M2 includes: Image encoder using CLIP model Extracting ISAR images Image features Text encoder using CLIP model Extract the corresponding description text Text features ,in This represents the parameters of the CLIP model; for the CLIP model, and They are all 512-dimensional vectors; Input multiple sets of ISAR images and explanatory text to obtain multiple pairs of image features and text features; Feature extraction is performed on all images and text in the training dataset; Using the extracted image and text features, a similarity score is calculated between the image and text features: in: This indicates the calculation of cosine similarity; when hour, Represents ISAR image Its corresponding explanatory text The similarity score of the features; when hour, Represents ISAR image With ISAR images Corresponding explanatory text The similarity score of the features; The module M3 includes: Randomly selected from each category A total of [number] images were obtained. Zhang Image Representing the number of categories, these images are one-hot encoded with their image features and ground truth labels as keys and values, respectively, to construct an initial cache model for continuous learning. This cache model contains a total of Each key-value pair is represented by a key portion consisting of image feature vectors. The value part, composed of the one-hot encoding of the label truth value, is represented as: During training, the key part of the cache model is fine-tuned. This allows for a better reflection of the image features of the dataset; During training, for each input ISAR image Image features are used to calculate their corresponding descriptive text for each category. The similarity score of the features yields a... dimensional vector That is, the CLIP model's own classifier for images The output; Calculated image features The similarity score between the image features and the cache model, i.e.: Through nonlinear transformation Take it from Mapped to , It is a hyperparameter, and then with Multiplying them together gives one dimensional vector That is, continuous learning of the caching model for images The output; The entire model for images The output is represented as ,in It's a hyperparameter; Use this output With images The true class labels are used to construct the cross-entropy loss, and gradient descent is used to optimize the key parameters of the continuously learning cached model.

5. The ISAR image recognition system based on CLIP and continuous learning according to claim 4, characterized in that, The module M1 includes: Receive ISAR image data and perform data augmentation, including: rotation, flipping, cropping, scaling, color transformation, noise injection, and blurring. Create corresponding explanatory text, which includes the ISAR image category and data augmentation method; Obtain ISAR image dataset With explanatory text dataset ,in Represents images in the dataset or text Total quantity Indicates the number of categories.

6. The ISAR image recognition system based on CLIP and continuous learning according to claim 4, characterized in that, The module M4 includes: for each ISAR image to be identified The final output is obtained by extracting features through the image encoder of the CLIP model, calculating the output of the classifier of the CLIP model itself, and calculating the output of the fine-tuned cache model. ,go through Function to obtain ISAR image The recognition results.

Images