Aerial Image Classification Method and Device Based on Enhanced Image Retrieval

Abstract

The present invention discloses an aerial image classification method and device based on enhanced image retrieval. The method includes steps of dataset construction, model construction, model training, and classification. In the backbone network, the feature embedding undergoes forward propagation through a fully connected layer to obtain a basic classification probability. In the retrieval branch, the feature embedding obtains corresponding text information in the retrieval table, and then through the forward propagation of the text encoder, the classification probability of the retrieval is obtained. The two classification probabilities are normalized and summed, and finally the final classification probability is obtained through an angular penalty module. Then, the angular penalty loss is calculated, and the network parameters are updated through backpropagation. Finally, a classification model is obtained, and the classification model is used for testing. The present invention can use retrieval to increase the classification probability predicted by the classification network, enhance the classification performance, can use the angular penalty loss, increase the penalty margin, and directly optimize the geodesic distance radian, so that each class has higher distinguishability and improves the accuracy of classification.

Description

Technical Field

[0001] The present invention relates to the technical fields of deep learning and computer vision, and particularly to an aerial image classification method and device based on enhanced image retrieval. Background Art

[0002] In recent years, with the popularization of aerial photography equipment such as drones, a large number of aerial images have been generated, and the recognition of the content of these images has a wide range of applications in fields such as agriculture and the ocean. How to accurately recognize these images has become an essential part of their application promotion.

[0003] Many current aerial image data are unbalanced and long-tailed distributed. There are more images of high-probability events, and conversely, there are fewer images of low-probability events. At the same time, due to changes in the aerial photography angle, shooting time, and location, images of the same content show various forms. This makes it difficult to train deep learning models in computer vision, and the recognition accuracy of images is relatively low.

[0004] Currently, in response to the above problems, most solutions in the prior art revolve around resampling and reweighting. Resampling is undersampling of the head categories and oversampling of the tail categories, and reweighting is assigning different weight values to the losses of different classes to minimize the training loss. Although it can improve the long-tailed problem, it will cause the small amount of data at the tail to be repeatedly learned, lacking sufficient sample differences and not being robust enough, while the large amount of data with sufficient differences at the head is not fully learned.

[0005] It can be seen that the methods in the prior art have the technical problem of low classification accuracy. Summary of the Invention

[0006] The present invention provides an aerial image classification method and device based on enhanced image retrieval to solve or at least partially solve the technical problem of low classification accuracy in the methods of the prior art.

[0007] To solve the above technical problem, in a first aspect of the present invention, an aerial image classification method based on enhanced image retrieval is provided, including:

[0008] S1: Construct a training data set for model training. The training data set includes aerial pictures and label information, and construct external storage data for retrieval based on the training data set. The external storage data includes aerial pictures and label information;

[0009] S2: Build an image classification model, which includes a feature extraction module, a backbone network, a retrieval branch, a normalization summation module, and an angular penalty module. Among them, the feature extraction module is used to extract features from the input image using a pre-trained encoder to obtain feature embeddings. The backbone network is used to process the feature embeddings of the training data using the encoder and perform forward propagation to obtain the basic classification probability. The retrieval branch is used to retrieve the feature embeddings of the external storage data in a pre-constructed retrieval table to obtain data similar to the external storage data, obtain the corresponding label information, and then obtain the retrieval classification probability through the forward propagation of the text encoder. The normalization summation module is used to perform normalization summation on the basic classification probability and the retrieval classification probability. The angular penalty module is used to obtain the final classification probability based on the result of the normalization summation;

[0010] S3: Train the constructed image classification model. During the training process, calculate the angular penalty loss and backpropagate to update the network parameters to obtain a trained image classification model;

[0011] S4: Use the trained image classification model to classify the image to be classified.

[0012] In one implementation, the encoders of the backbone network and the retrieval branch use the ViT network model, and based on the transformer and attention mechanisms, obtain feature embeddings corresponding to the input data.

[0013] In one implementation, the processing process of the retrieval branch includes:

[0014] Use the k-NN nearest neighbor algorithm in the pre-constructed retrieval table to obtain the indices of k similar data;

[0015] Obtain the label information of the similar data according to the indices;

[0016] Input the label information of the similar data into the text encoder, and encode the label information into a classification probability with the same shape as the basic classification probability through the forward propagation of the text encoder as the retrieval classification probability.

[0017] In one implementation, after obtaining the indices of the data similar to the external storage data, the method further includes:

[0018] Discard the first result in the indices.

[0019] In one implementation, the angular penalty module includes a fully connected layer with normalized weights, which normalizes the input data so that the output value is only related to the angle. The specific method is:

[0020]

[0021] fc (x) represents the basic classification probability obtained by the backbone network, f r (x) represents the retrieval classification probability obtained by the retrieval branch.

[0022] In one implementation, the calculation method of the angular penalty loss in step S3 is as follows:

[0023]

[0024] where N is the batch size set for training, n is the number of classes, y i is the class of the i-th sample, θ j is the angle of the j-th class, is the angle of the class to which the i-th sample belongs, s is the scaling value, and β is a random value on the β distribution.

[0025] Based on the same inventive concept, the second aspect of the present invention provides an aerial image classification device based on image retrieval enhancement, including:

[0026] A dataset construction module for constructing a training dataset for model training. The training dataset includes aerial pictures and label information, and an external storage data for retrieval is constructed based on the training dataset. The external storage data includes aerial pictures and label information;

[0027] A model construction module for constructing an image classification model. The model includes a feature extraction module, a backbone network, a retrieval branch, a normalization summation module, and an angular penalty module. Among them, the feature extraction module is used to extract features of the input picture by using a pre-trained encoder to obtain feature embeddings. The backbone network is used to process the feature embeddings of the training data by using the encoder and perform forward propagation to obtain the basic classification probability. The retrieval branch is used to retrieve the feature embeddings of the external storage data in a pre-constructed retrieval table to obtain data similar to the external storage data, obtain the corresponding label information, and then obtain the retrieval classification probability through the forward propagation of the text encoder. The normalization summation module is used to perform normalization summation on the basic classification probability and the retrieval classification probability. The angular penalty module is used to obtain the final classification probability according to the result of the normalization summation;

[0028] A model training module for training the constructed image classification model. During the training process, the angular penalty loss is calculated and the network parameters are updated by backpropagation to obtain a trained image classification model;

[0029] A classification module for classifying the image to be classified by using the trained image classification model.

[0030] Based on the same inventive concept, the third aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed, the method described in the first aspect is implemented.

[0031] Based on the same inventive concept, the fourth aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method described in the first aspect is implemented.

[0032] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows:

[0033] An aerial image classification method based on image retrieval enhancement provided by the present invention constructs an image classification model. The retrieval branch can retrieve data similar to the input data, and then obtain a retrieval classification probability. The retrieval classification probability can be used to enhance the basic classification probability, thereby enhancing the classification performance. And by using the angular penalty loss and increasing the penalty margin, the geodesic distance radian is directly optimized, so that each class has higher distinguishability and the classification accuracy is improved. Description of the Drawings

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0035] Figure 1 It is a schematic diagram of the overall process of the aerial image classification method based on image retrieval enhancement in the embodiments of the present invention;

[0036] Figure 2 It is a schematic diagram of the structure of the encoder in the embodiments of the present invention;

[0037] Figure 3 It is a schematic diagram of the structure of the backbone network in the embodiments of the present invention;

[0038] Figure 4 It is a network structure diagram of the retrieval branch in the embodiments of the present invention. Detailed Embodiments

[0039] The present invention provides an aerial image classification method based on enhanced image retrieval, mainly including: Step 1, constructing a training data set for model training, the training data set includes aerial pictures and label information, and constructing external storage data for retrieval; Step 2, constructing an image classification model, the training data is subjected to feature extraction by a pre-trained encoder to obtain feature embeddings, and then enters the backbone network and the retrieval branch respectively. Among them, in the backbone network, the feature embeddings obtain basic classification probabilities through the forward propagation of the fully connected layer; in the retrieval branch, the feature embeddings are retrieved in the retrieval table to obtain corresponding text information, where the retrieval table is obtained by the training data through the same encoder as the feature extraction, and then the classification probability of the retrieval is obtained through the forward propagation of the text encoder; then the two classification probabilities are normalized and summed, and through the angular penalty module, the final classification probability is obtained; Step 3, training the constructed image classification model, calculating the angular penalty loss during the training process, and backpropagating to update the network parameters to obtain a trained image classification model; Step 4: Using the trained image classification model to classify the images to be classified.

[0040] Among them, the overall process of the aerial image classification method is as Figure 1 shown. The present invention proposes a new image classification method, which can use retrieval to increase the classification probability predicted by the classification network, enhance the classification performance, can use the angular penalty loss, increase the penalty margin, directly optimize the geodesic distance radian, so that each class has higher distinguishability and improves the classification accuracy.

[0041] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0042] Embodiment 1

[0043] The embodiment of the present invention provides an aerial image classification method based on enhanced image retrieval, including:

[0044] S1: Constructing a training data set for model training, the training data set includes aerial pictures and label information, and constructing external storage data for retrieval based on the training data set, the external storage data includes aerial pictures and label information;

[0045] S2: Build an image classification model, which includes a feature extraction module, a backbone network, a retrieval branch, a normalization summation module, and an angular penalty module. Among them, the feature extraction module is used to extract features from the input image using a pre-trained encoder to obtain feature embeddings. The backbone network is used to process the feature embeddings of the training data using the encoder and perform forward propagation to obtain basic classification probabilities. The retrieval branch is used to retrieve the feature embeddings of the external storage data in a pre-constructed retrieval table to obtain data similar to the external storage data, obtain the corresponding label information, and then obtain retrieval classification probabilities through the forward propagation of the text encoder. The normalization summation module is used to perform normalization summation on the basic classification probabilities and the retrieval classification probabilities. The angular penalty module is used to obtain the final classification probability based on the result of the normalization summation;

[0046] S3: Train the constructed image classification model. During the training process, calculate the angular penalty loss and backpropagate to update the network parameters to obtain a trained image classification model;

[0047] S4: Use the trained image classification model to classify the image to be classified.

[0048] Regarding the technical problem in the prior art that a small amount of data at the tail is repeatedly learned through resampling and reweighting methods, lacking sufficient sample differences and being not robust enough, while a large amount of data with sufficient differences at the head cannot be fully learned, resulting in low classification accuracy. The present invention uses retrieval to enhance classification, uses external storage, clearly stores the tail knowledge, and through the label includes unconnected data sources (i.e., obtaining labels during retrieval, including class labels not included in the current batch), enabling the head classes to be well learned in the backbone network, while the tail classes are enhanced by the retrieval branch, so as to enhance the learning of the tail data while not reducing the learning of the head data, and enhance the classification effect. At the same time, an improved angular penalty loss function is added to replace the cross-entropy loss function in normal classification to improve the classification accuracy for the same content with variability and differences.

[0049] Specifically, the external storage data used for retrieval is a part of the training data or equivalent to the training data.

[0050] The pre-constructed retrieval table in step S3 is obtained by the same encoder as the feature extraction module for the training data, but the process of obtaining the retrieval table does not participate in the backpropagation of the gradient.

[0051] Before entering the backbone network and the retrieval branch, first perform feature extraction through the feature extraction module, such as Figure 2The structure of the encoder used in the feature extraction module is shown in Figure 1. The input data first passes through a convolution layer, then is flattened in height and width, followed by the concat class token, and then the position embedding addition operation. After Dropout, 12 stacked encoding blocks are input to obtain the feature embedding z. q .

[0052] The feature embedding input into the backbone network is normalized by layer, and the token of the first class is extracted to obtain the classification probability of the backbone network. Figure 3 Schematic diagram of the structure of the backbone network in an embodiment of the present invention, Layer norm represents the normalization layer, Multi-headAttention represents the multi-head attention mechanism, and MLP represents the multi-layer perceptron.

[0053] In one embodiment, the encoder of the backbone network and the retrieval branch uses the ViT network model based on the transformer and attention mechanism to obtain feature embedding corresponding to the input data.

[0054] In one embodiment, the process of retrieving a branch includes:

[0055] Using the k-NN nearest neighbor algorithm in the pre-built search table to obtain the index of k similar data;

[0056] Get label information of similar data based on the index;

[0057] The label information of similar data is input into the text encoder, and the label information is encoded into a classification probability of the same shape as the basic classification probability through the forward propagation of the text encoder as the retrieval classification probability.

[0058] Specifically, the text encoder can be a Bert model. The network structure diagram of the retrieval branch is as follows Figure 4 As shown in the figure, all the retrieval data are passed through the ViT encoder to obtain the retrieval table Z. This process does not participate in the back propagation of the gradient. Then the features of the training data are embedded in the retrieval table for retrieval, the cosine similarity is calculated, and the k-NN neighbor method is used to obtain the label information of k similar features. Finally, these label information are input into the text encoder to obtain the classification probability of the retrieval branch.

[0059] In one embodiment, after obtaining the index of data similar to the externally stored data, the method further includes:

[0060] Discard the first result in the index.

[0061] Specifically, since the first result in the index results is usually the original image, this causes the text encoder to impose inappropriate weights on the first retrieved label when creating predictions. Therefore, after obtaining the index, for the first result in the index (the label of the first data).

[0062] In one embodiment, the angle penalty module includes a fully connected layer that normalizes the weights, normalizing the input data so that the output value is only related to the angle. The specific method is as follows:

[0063]

[0064] f c (x) represents the basic classification probability obtained by the backbone network, and f r (x) represents the retrieval classification probability obtained by the retrieval branch.

[0065] Specifically, the angle penalty module mainly transforms the output features into features that are only related to the angle, and then operates on the angle to make the angle between each class larger and more distinguishable, and also more compact within the same class.

[0066] The specific operation is as follows: Perform the calculation of W T x + b for the fully connected layer. First, set b to 0, and then transform W T x into the form of ‖W‖‖x‖cosθ. Therefore, this module is mainly a fully connected layer with a bias set to 0 and L2 regularization of the weight values, and L2 regularization is also performed on the input. For the θ of the label class, a margin penalty is added, that is, the angle is changed to θ + β, where β is a continuous β distribution, and the random value taken on this distribution is added at this time. At the same time, a scaling factor s is added to project all features onto the same hypersphere to obtain the final classification probability.

[0067] A penalty margin is added to the angle penalty module. The value of the penalty margin comes from the random value of β. In this embodiment, β is selected as the β distribution, so that there is higher separability between different classes, the tightness within the same class is improved, and the classification boundary is maximized in the angle space.

[0068]

[0069] is the classification probability output by the neural network for the i-th sample in the j-th class.

[0070] In one embodiment, the calculation method of the angle penalty loss in step S3 is as follows:

[0071]

[0072] where N is the batch size set for training, n is the number of classes, and y i is the class of the i-th sample, and θ j is the angle of the j-th class, is the angle of the class to which the i-th sample belongs, s is the scaling value, and β is a random value on the β distribution.

[0073] In the specific implementation process, during training, the angular penalty loss is calculated, and the network parameters are updated through backpropagation. Finally, a classification model is obtained for classification testing.

[0074] The angular penalty loss calculates the cross-entropy loss function after improving the finally obtained classification probability. The added angular margin is only added to the corresponding label class, and the other classes remain unchanged at the original angle. Then, after providing the update, the image encoder and the text encoder are updated simultaneously.

[0075] The beneficial effects of the present invention are as follows: The present invention proposes a classification method enhanced by image retrieval. Through external stored data, the text information (label information) of similar pictures can be obtained, and similar classification probabilities are generated through the text encoder to enhance the tail classes of the image data, thereby improving the classification accuracy. The present invention uses an angular penalty loss function to directly optimize the geodesic distance in radians, increase the penalty margin, make the classes highly separable from each other, and increase the tightness within the same class.

[0076] Embodiment 2

[0077] Based on the same inventive concept, this embodiment provides an aerial image classification device enhanced by image retrieval, including:

[0078] A dataset construction module for constructing a training dataset for model training. The training dataset includes aerial pictures and label information, and an external stored data for retrieval is constructed based on the training dataset. The external stored data includes aerial pictures and label information;

[0079] A model construction module for constructing an image classification model, which includes a feature extraction module, a backbone network, a retrieval branch, a normalization summation module, and an angle penalty module. Among them, the feature extraction module is used to extract features from the input image using a pre-trained encoder to obtain feature embeddings. The backbone network is used to process the feature embeddings of the training data using the encoder and perform forward propagation to obtain the basic classification probability. The retrieval branch is used to retrieve the feature embeddings of the external storage data in a pre-constructed retrieval table to obtain data similar to the external storage data, obtain the corresponding label information, and then obtain the retrieval classification probability through the forward propagation of the text encoder. The normalization summation module is used to perform normalization summation on the basic classification probability and the retrieval classification probability. The angle penalty module is used to obtain the final classification probability based on the result of the normalization summation;

[0080] A model training module for training the constructed image classification model. During the training process, the angle penalty loss is calculated, and the network parameters are updated through backpropagation to obtain a trained image classification model;

[0081] A classification module for classifying the image to be classified using the trained image classification model.

[0082] Since the device introduced in the second embodiment of the present invention is the device used for aerial image classification enhanced by image retrieval in the first embodiment of the present invention, based on the method introduced in the first embodiment of the present invention, those skilled in the art can understand the specific structure and deformation of the device, so it will not be elaborated here. Any device used in the method of the first embodiment of the present invention belongs to the scope protected by the present invention.

[0083] Embodiment Three

[0084] Based on the same inventive concept, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed, it implements the method described in Embodiment One.

[0085] Since the computer-readable storage medium introduced in the third embodiment of the present invention is the computer-readable storage medium used for aerial image classification enhanced by image retrieval in the first embodiment of the present invention, based on the method introduced in the first embodiment of the present invention, those skilled in the art can understand the specific structure and deformation of the computer-readable storage medium, so it will not be elaborated here. Any computer-readable storage medium used in the method of the first embodiment of the present invention belongs to the scope protected by the present invention.

[0086] Embodiment Four

[0087] Based on the same inventive concept, the present application also provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the above program, the method in Embodiment 1 is implemented.

[0088] Since the computer device introduced in Embodiment 4 of the present invention is the computer device used to implement the aerial image classification enhanced by image retrieval in Embodiment 1 of the present invention, based on the method introduced in Embodiment 1 of the present invention, those skilled in the art can understand the specific structure and variations of this computer device, so it will not be elaborated here. Any computer device adopted by the method in Embodiment 1 of the present invention falls within the scope of protection of the present invention.

[0089] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0090] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0091] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they know the basic creative concepts. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0092] Obviously, those skilled in the art can make various changes and variations to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if these modifications and variations of the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these changes and variations.

Claims

1. Aerial image classification method based on enhanced image retrieval, characterized in that Including: S1: Construct a training data set for model training. The training data set includes aerial images and label information. Based on the training data set, construct external storage data for retrieval. The external storage data includes aerial images and label information; S2: Construct an image classification model. The model includes a feature extraction module, a backbone network, a retrieval branch, a normalization summation module, and an angle penalty module. Among them, the feature extraction module is used to extract features of the input image using a pre-trained encoder to obtain feature embeddings. The backbone network is used to process the feature embeddings of the training data using the encoder and perform forward propagation to obtain a basic classification probability. The retrieval branch is used to retrieve the feature embeddings of the external storage data in a pre-constructed retrieval table to obtain data similar to the external storage data, obtain the corresponding label information, and then obtain a retrieval classification probability through the forward propagation of the text encoder. The normalization summation module is used to perform normalization summation on the basic classification probability and the retrieval classification probability. The angle penalty module is used to obtain the final classification probability according to the result of the normalization summation. Among them, the processing process of the retrieval branch includes: Using the k-NN nearest neighbor algorithm in a pre-constructed retrieval table to obtain the indices of k similar data; Obtaining the label information of the similar data according to the indices; Inputting the label information of the similar data into the text encoder, and encoding the label information into a classification probability with the same shape as the basic classification probability through the forward propagation of the text encoder as the retrieval classification probability The angle penalty module includes a fully connected layer with normalized weights, which normalizes the input data so that the output value is only related to the angle. The specific method is: Indicates the basic classification probability obtained by the backbone network, Indicates the retrieval classification probability obtained by the retrieval branch; S3: Train the constructed image classification model. During the training process, calculate the angle penalty loss and backpropagate to update the network parameters to obtain a trained image classification model; S4: Use the trained image classification model to classify the image to be classified.

2. The aerial image classification method based on enhanced image retrieval as claimed in claim 1, wherein The encoders of the backbone network and the retrieval branch use the ViT network model, and based on the transformer and attention mechanisms, obtain feature embeddings corresponding to the input data.

3. The aerial image classification method based on enhanced image retrieval as claimed in claim 1, wherein, After obtaining the indices of the data similar to the external storage data, the method further includes: Discarding the first result in the indices.

4. The aerial image classification method based on enhanced image retrieval as claimed in claim 1, wherein, The calculation method of the angle penalty loss in step S3 is: Among them, is the batch size set for training, is the number of classes, is the class of the i-th sample, is the angle of the j-th class, is the angle of the class to which the i-th sample belongs, is the scaling value, is at random value taken on the distribution.

5. An aerial image classification device based on enhanced image retrieval, characterized in that, Including: A data set construction module, which is used to construct a training data set for model training. The training data set includes aerial images and label information. Based on the training data set, construct external storage data for retrieval. The external storage data includes aerial images and label information; A model construction module for constructing an image classification model, which includes a feature extraction module, a backbone network, a retrieval branch, a normalization summation module, and an angle penalty module. Among them, the feature extraction module is used to extract features from the input picture by using a pre-trained encoder to obtain feature embeddings. The backbone network is used to process the feature embeddings of the training data by using the encoder and perform forward propagation to obtain basic classification probabilities. The retrieval branch is used to retrieve the feature embeddings of the external stored data in a pre-constructed retrieval table to obtain data similar to the external stored data, obtain the corresponding label information, and then obtain the retrieval classification probability through the forward propagation of the text encoder. The normalization summation module is used to perform normalization summation on the basic classification probability and the retrieval classification probability. The angle penalty module is used to obtain the final classification probability according to the result of the normalization summation. Among them, the processing process of the retrieval branch includes: Obtaining the indexes of k similar data by using the k-NN nearest neighbor algorithm in the pre-constructed retrieval table; Obtaining the label information of the similar data according to the indexes; Inputting the label information of the similar data into the text encoder, and encoding the label information into a classification probability with the same shape as the basic classification probability through the forward propagation of the text encoder as the retrieval classification probability The angle penalty module includes a fully connected layer with normalized weights, which normalizes the input data so that the output value is only related to the angle. The specific method is: Represents the basic classification probability obtained by the backbone network, Represents the retrieval classification probability obtained by the retrieval branch; A model training module for training the constructed image classification model. During the training process, the angle penalty loss is calculated and the network parameters are updated by backpropagation to obtain a trained image classification model; A classification module for classifying the image to be classified by using the trained image classification model.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed, it implements the method described in any one of claims 1 to 4.

7. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1 to 4.

Images