Image classification model training method, apparatus, device, and medium
By combining contrastive learning loss and local entropy regularization loss, the problem of catastrophic forgetting in image classification model training is solved, improving the model's ability to recognize new categories of images and its overall accuracy.
Patent Information
- Application Number
- CN202410788532.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-18
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2044-06-18
AI Technical Summary
In existing technologies, image classification models are prone to catastrophic forgetting during training, resulting in low image classification accuracy after training. This is especially true when using unlabeled data in massive image datasets, making it difficult to effectively identify multiple image categories.
By acquiring the original image set and determining the view, image feature vectors are calculated. The model parameters are updated using contrastive learning loss, classification loss, and local entropy regularization loss to ensure that the model retains known category knowledge when learning new categories and avoids forgetting.
This improves the image classification model's ability to extract features and classify different training images, enhances the model's accuracy in identifying new image categories, avoids catastrophic forgetting, and improves the overall accuracy of image classification.
Smart Images

Figure CN118823429B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, specifically to an image classification model training method, apparatus, device, and medium. Background Technology
[0002] With the development of artificial intelligence technology, various tasks can be performed using artificial intelligence models. For example, image classification models can be used to perform image classification.
[0003] To enable image classification models to recognize multiple categories of images and implement generalized category discovery, related technologies often rely on manual image data annotation. However, this is time-consuming and labor-intensive, and can only annotate a portion of the data. In the real world, there are often massive amounts of image data involved. During the training of image classification models, a large amount of unlabeled image data is also used, and then the image classification model is trained using both labeled and unlabeled image data.
[0004] During the training of an image classification model, learning parameters corresponding to previously labeled image category data are passed to learning parameters corresponding to new image categories. However, this process can easily lead to catastrophic forgetting in the image classification model, ultimately resulting in low image classification accuracy after training. Summary of the Invention
[0005] This application provides an image classification model training method, apparatus, device, and medium. It can accurately calculate the model loss during the training process, and use this model loss to iteratively train the image classification model, thereby avoiding catastrophic forgetting during training and resulting in a trained image classification model with high image classification accuracy.
[0006] According to one aspect of this application, an image classification model training method is provided, comprising:
[0007] Obtain the original image set, determine at least two views corresponding to each original image in the original image set, and use the views as training images to obtain the training image set, which contains labeled training images and unlabeled training images;
[0008] Obtain the image feature vector corresponding to each training image, and determine the contrastive learning loss corresponding to the image classification model based on the image feature vector;
[0009] The image feature vector is input into the image classification model, which outputs the image class probability corresponding to each training image. Based on the image class probability and the labels of the labeled training images, the classification loss corresponding to the image classification model is determined.
[0010] The target training image is determined from the unlabeled training images based on the image category probability. The image category of the target training image is the known image category corresponding to the training image.
[0011] Based on the image category probabilities corresponding to the target training image, the local entropy regularization loss corresponding to the image classification model is determined;
[0012] The model loss corresponding to the image classification model is determined by contrastive learning loss, classification loss and local entropy regularization loss, and the model parameters of the image classification model are updated according to the model loss to iteratively train the image classification model.
[0013] According to one aspect of this application, an image classification model training apparatus is provided, comprising:
[0014] The acquisition module is used to acquire the original image set, determine at least two views corresponding to each original image in the original image set, and determine the views as training images to obtain the training image set, which contains labeled training images and unlabeled training images.
[0015] The first loss module is used to obtain the image feature vector corresponding to each training image and determine the contrastive learning loss corresponding to the image classification model based on the image feature vector.
[0016] The second loss module is used to input the image feature vector into the image classification model, output the image class probability corresponding to each training image, and determine the classification loss corresponding to the image classification model based on the image class probability and the labels of the labeled training images.
[0017] The determination module is used to determine the target training image from the unlabeled training images based on the image category probability. The image category of the target training image is the known image category corresponding to the training image.
[0018] The third loss module is used to determine the local entropy regularization loss corresponding to the image classification model based on the image category probability corresponding to the target training image.
[0019] The fourth loss module is used to determine the model loss corresponding to the image classification model based on the contrastive learning loss, classification loss, and local entropy regularization loss, and to update the model parameters of the image classification model based on the model loss in order to iteratively train the image classification model.
[0020] In some implementations, the first loss module includes a feature extraction submodule, an unsupervised submodule, a supervised submodule, and a determination submodule;
[0021] The feature extraction submodule is used to input the image feature vector into the multilayer perceptron and output the target image feature vector corresponding to each training image;
[0022] The unsupervised submodule is used to calculate the unsupervised contrastive learning loss corresponding to the image classification model based on the feature vector of the target image and the first temperature coefficient.
[0023] The supervised submodule is used to determine the supervised contrastive learning loss corresponding to the image classification model based on the second temperature coefficient and the feature vector of the target image of the labeled training image.
[0024] The determination submodule is used to determine the contrastive learning loss corresponding to the image classification model based on the unsupervised contrastive learning loss and the supervised contrastive learning loss.
[0025] In some implementations, an unsupervised submodule is used to identify associated training images generated based on the same original image in the training images;
[0026] The first distance between the associated training images is determined based on the first temperature coefficient and the feature vector of the target image of the associated training images;
[0027] Based on the first temperature coefficient and the feature vector of the target image of the training image, a second distance between any training image and an unrelated training image is determined;
[0028] The unsupervised contrastive learning loss corresponding to the image classification model is determined based on the first distance and the second distance.
[0029] In some implementations, there is a supervisory submodule for identifying training images with the same label among the labeled training images;
[0030] Based on the second temperature coefficient and the feature vector of the target image of the training image with the same label, the third distance between the training images with the same label is determined.
[0031] Based on the second temperature coefficient and the target image feature vector of the labeled training images, a fourth distance between any labeled training image and other labeled training images is determined.
[0032] The supervised contrastive learning loss corresponding to the image classification model is determined based on the third and fourth distances.
[0033] In some implementations, the second loss module includes a first sub-loss module, a second sub-loss module, a constraint sub-module, and a third sub-loss module;
[0034] The first sub-loss module is used to determine the first cross-entropy loss corresponding to the image classification model based on the image category probability of the unlabeled training images.
[0035] The second sub-loss module is used to determine the second cross-entropy loss corresponding to the image classification model based on the image class probabilities and labels of the labeled training images.
[0036] The constraint submodule is used to determine the average entropy regularization constraint corresponding to the image classification model based on the image category probability of the training images.
[0037] The third sub-loss module is used to determine the classification loss corresponding to the image classification model based on the first cross-entropy loss, the second cross-entropy loss, and the average entropy regularization constraint.
[0038] In some implementations, a constraint submodule is used to identify associated training images generated based on the same original image in the training images;
[0039] Determine the sum of the probabilities of image categories associated with the training images;
[0040] Determine the number of training images corresponding to the image classification model in the current training process;
[0041] The average entropy regularization constraint corresponding to the image classification model is determined based on the sum and quantity.
[0042] In some implementations, the third sub-loss module is used to determine the weight coefficients corresponding to the average entropy regularization constraint, and multiply the weight coefficients by the average entropy regularization constraint to obtain the target average entropy regularization constraint.
[0043] The classification loss corresponding to the training image is determined based on the first cross-entropy loss, the second cross-entropy loss, and the target average entropy regularization constraint.
[0044] In some implementations, the third sub-loss module is used to determine the two-view divergence constraint corresponding to the associated training image based on the quantity and the image class probability of the associated training image;
[0045] The classification loss corresponding to the training image is determined based on the first cross-entropy loss, the second cross-entropy loss, the target average entropy regularization constraint, and the two-view divergence constraint.
[0046] In some implementations, the determining module is used to determine candidate training images with a higher probability than a preset threshold among unlabeled training images based on the image category probability.
[0047] The target image category of the candidate training images is determined based on the image category probability of the candidate training images;
[0048] Candidate training images whose target image category is a known image category are selected as target training images.
[0049] The third loss module is used to determine the entropy regularization constraint corresponding to the image classification model based on the third temperature coefficient and the image category probability of the target training image.
[0050] Determine the number of training images corresponding to the image classification model in the current training process;
[0051] The local entropy regularization loss of the image classification model is determined based on the entropy regularization constraint corresponding to the target training image and the number of targets.
[0052] The fourth loss module is used to multiply the sum of the contrastive learning loss and the classification loss by the weight of the first loss to obtain the first result;
[0053] Multiplying the local entropy regularization loss by the second loss weight yields the second result.
[0054] Add the first result and the second result to obtain the model loss corresponding to the image classification model.
[0055] According to one aspect of this application, a computer-readable storage medium is provided, which stores a plurality of instructions adapted for loading by a processor to execute the image classification model training method provided in the embodiments of this application.
[0056] According to one aspect of this application, a computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the image classification model training method provided in the embodiments of this application.
[0057] In this embodiment, by acquiring an original image set, at least two views corresponding to each original image in the original image set are determined, and these views are used as training images to obtain a training image set, which includes labeled training images and unlabeled training images. The image feature vector corresponding to each training image is obtained, and the contrastive learning loss corresponding to the image classification model is determined based on the image feature vector. The image feature vector is input into the image classification model, and the image class probability corresponding to each training image is output. The classification loss corresponding to the image classification model is determined based on the image class probability and the labels of the labeled training images. The target training image is determined from the unlabeled training images based on the image class probability, and the image class of the target training image is the known image class corresponding to the training image. The local entropy regularization loss corresponding to the image classification model is determined based on the image class probability corresponding to the target training image. The model loss corresponding to the image classification model is determined based on the contrastive learning loss, the classification loss, and the local entropy regularization loss. The model parameters of the image classification model are updated based on the model loss to iteratively train the image classification model. In this embodiment, by determining the contrastive learning loss corresponding to the image classification model, the image classification model's ability to extract image features from different training images can be improved, as can its classification ability. By determining the image category probability of the target training image with a known image category in unlabeled training images, the local entropy regularization loss corresponding to the image classification model is determined. This helps the image classification model retain knowledge related to known image categories when learning new image categories, avoiding catastrophic forgetting during training. Furthermore, determining the classification loss corresponding to the training images helps improve the image classification model's ability to recognize new image categories in the training images. Therefore, the model loss determined by the contrastive learning loss, classification loss, and local entropy regularization loss can more accurately guide the training direction of the image classification model, thereby avoiding catastrophic forgetting during training and resulting in a higher image classification accuracy after training.
[0058] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description
[0059] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0060] Figure 1 This is a system architecture diagram of the image classification model training method provided in the embodiments of this application;
[0061] Figure 2 This is a schematic diagram of an image classification scenario provided in an embodiment of this application;
[0062] Figure 3 This is a flowchart illustrating the image classification model training method provided in an embodiment of this application;
[0063] Figure 4 This is a schematic diagram illustrating the steps included in step 220 provided in the embodiments of this application;
[0064] Figure 5 This is a flowchart illustrating the steps included in step 230 of the embodiments of this application;
[0065] Figure 6 This is a flowchart illustrating the steps included in step 250 provided in an embodiment of this application;
[0066] Figure 7 This is another flowchart illustrating the image classification model training method provided in this application embodiment;
[0067] Figure 8 This is a schematic diagram of the structure of the image classification model training device provided in the embodiments of this application;
[0068] Figure 9 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Detailed Implementation
[0069] To enable those skilled in the art to better understand the solutions of this application, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0070] It is understood that in the specific embodiments of this application, the text data and other related data are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with relevant laws, regulations and standards.
[0071] It should be noted that while some processes described in the specification, claims, and accompanying drawings contain multiple steps that appear in a specific order, it should be clearly understood that these steps may not be performed in the order they appear herein, or may be performed in parallel. The step numbers are merely used to distinguish different steps and do not represent any particular order of execution. Furthermore, descriptions such as "first," "second," or "objective" in this document are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.
[0072] Before providing a further detailed description of the embodiments of this disclosure, the terms and concepts used in these embodiments are explained, and they are subject to the following interpretations:
[0073] Catastrophic forgetting is a machine learning phenomenon where a model suddenly loses knowledge and skills learned for previously learned tasks when learning a new one. This phenomenon typically occurs in neural network models, especially in continuous learning scenarios, where the model needs to constantly adapt to new tasks without forgetting old ones.
[0074] The cause of catastrophic forgetting is primarily related to the weight update mechanism of neural networks. When a model learns a new task, its weights are adjusted to minimize the loss function of that task. However, this adjustment may disrupt the weight configuration learned from previous tasks, leading to a decline in the model's performance on older tasks.
[0075] Local Entropy Regularization (LER) is a technique used to enhance the generalization ability of a model, particularly in deep learning and reinforcement learning. It is achieved by adding a regularization term to the loss function, which encourages the model to maintain high entropy values in local regions of the input space.
[0076] Entropy is a metric for measuring information content and describing the distribution of data. In machine learning, entropy can be used to measure the predictive uncertainty of a model. The core idea of local entropy regularization is to encourage the model to have higher uncertainty within the local neighborhood of the input data during model training, thereby improving the model's ability to generalize to new data.
[0077] Dual-view Kullback–Leiblerdivergence constraint: Also known as dual-view KL divergence constraint, this is a method used to ensure consistency between two different views (or representations), often used in scenarios such as multi-view learning and collaborative training. In this method, divergence is used to measure the difference between the two views, and consistency is promoted by adding a divergence constraint term to the loss function.
[0078] Unsupervised contrastive loss is a loss function used for unsupervised learning tasks, particularly widely applied in self-supervised learning and feature extraction. Its core idea is to learn the intrinsic structure and representation of data by comparing similar and dissimilar samples.
[0079] The basic principle of unsupervised contrastive learning loss is that, for any sample in the input dataset, we hope that the model can distinguish whether the sample is similar to other samples.
[0080] Supervised contrastive loss is a contrastive learning loss function that incorporates ideas from supervised learning. Unlike unsupervised contrastive loss, supervised contrastive loss utilizes known label information to guide the model in learning more discriminative feature representations. This approach is particularly useful in classification tasks because it leverages label information to enhance the learning process.
[0081] In supervised contrastive learning loss, positive pairs typically refer to sample pairs belonging to the same class, while negative pairs refer to sample pairs belonging to different classes. By minimizing the distance between positive pairs and maximizing the distance between negative pairs, the model can learn more accurate and discriminative feature representations.
[0082] The general form of supervised contrastive learning loss is similar to that of unsupervised contrastive learning loss, but the label information of the samples is taken into account when calculating the loss.
[0083] Cross-entropy loss: The cross-entropy loss function is a commonly used loss function in machine learning and deep learning, especially in classification problems. It measures the difference between two probability distributions and is typically used to measure the difference between the model output and the true label. In binary classification problems, assuming the model's prediction value represents the probability that a sample belongs to class 1, the true label is 0 or 1.
[0084] Generalized category discovery (GCL) is a machine learning method used to discover latent class structures in data under unsupervised or semi-supervised conditions. This method aims to identify data points with similar characteristics and group them into a single class. GCL has wide applications, including image recognition, natural language processing, and bioinformatics.
[0085] First, we will describe some of the technical problems that exist in the relevant technologies.
[0086] With the development of artificial intelligence technology, various tasks can be performed using artificial intelligence models. For example, image classification models can be used to perform image classification.
[0087] To enable image classification models to recognize multiple categories of images and implement generalized category discovery, related technologies often rely on manual image data annotation. However, this is time-consuming and labor-intensive, and can only annotate a portion of the data. In the real world, there are often massive amounts of image data involved. During the training of image classification models, a large amount of unlabeled image data is also used, and then the image classification model is trained using both labeled and unlabeled image data.
[0088] During the training of an image classification model, learning parameters corresponding to previously labeled image category data are passed to learning parameters corresponding to new image categories. However, this process can easily lead to catastrophic forgetting in the image classification model, ultimately resulting in low image classification accuracy after training.
[0089] Therefore, image classification models trained using relevant technologies often suffer from low image classification accuracy.
[0090] To address the aforementioned technical problems, this application proposes an image classification model training method. By determining the contrastive learning loss corresponding to the image classification model, the method improves the model's ability to extract image features from different training images and enhances its classification capabilities. By determining the image category probabilities of target training images with known categories in unlabeled training images, the local entropy regularization loss corresponding to the image classification model is determined. This helps the model retain knowledge related to known image categories when learning new image categories, avoiding catastrophic forgetting during training. Furthermore, determining the classification loss corresponding to the training images helps improve the model's ability to recognize new image categories within the training images. Therefore, the model loss determined through contrastive learning loss, classification loss, and local entropy regularization loss can more accurately guide the training direction of the image classification model, thereby avoiding catastrophic forgetting during training and resulting in a highly accurate image classification model after training.
[0091] The image classification model training method, apparatus, computer equipment, and storage medium provided in the embodiments of this application will be described in detail later.
[0092] Please see Figure 1 , Figure 1 This is a system architecture diagram of the image classification model training method provided in this application embodiment. It includes a terminal 140, an Internet 130, a gateway 120, a server 110, etc.
[0093] Terminal 140 includes, but is not limited to, mobile phones, computers, smart voice interaction devices, smart home appliances, vehicle terminals, and aircraft. This application embodiment can be applied to various scenarios, including but not limited to cloud technology and artificial intelligence. Furthermore, it can be a single device or a collection of multiple devices. For example, multiple desktop computers connected to each other via a local area network, sharing a single monitor, etc., can work collaboratively to form a terminal 140. Terminal 140 can communicate with the Internet 130 via wired or wireless means to exchange data.
[0094] Server 110 refers to a computer system that can provide certain services to terminal 140. Compared to ordinary terminal 140, server 110 has higher requirements in terms of stability, security, and performance. Server 110 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.
[0095] Gateway 120, also known as an internetwork connector or protocol converter, is a computer system or device that acts as a translator, enabling network interconnection at the transport layer. It bridges the gap between two systems using different communication protocols, data formats, languages, or even completely different architectures. Gateways can also provide filtering and security functions. Messages sent from terminal 140 to server 110 are forwarded to the corresponding server 110 via gateway 120. Messages sent from server 110 to terminal 140 are also forwarded to the corresponding terminal 140 via gateway 120.
[0096] The image classification model trained by the image classification model training method in the embodiments of this application can be applied to a variety of scenarios, such as autonomous driving, smart cities, healthcare, digital security, media, and so on.
[0097] Please refer to the following: Figure 2 , Figure 2 This is a schematic diagram of an image classification scenario provided in an embodiment of this application.
[0098] The image classification model is a model trained using the image classification model training method provided in the embodiments of this application. During the training process, the image classification model aims to discover categories in a generalized way and achieves image classification of labeled and unlabeled images.
[0099] In a batch of images that need to be classified, the image features of different images can be input into a trained image classification model, such as image 1, image 2, image 3, ... image N. For example, an existing trained image feature extraction model can be used to initially extract the image features corresponding to each image. Then, the image features corresponding to each image can be input into the trained image classification model. The trained image classification model can output the probability of the image category corresponding to each image, and then the image category corresponding to each image can be determined based on the probability.
[0100] For example, image 1 corresponds to image category 1, image 2 corresponds to image category 2, and image N corresponds to image category N. Among the image categories corresponding to the images to be identified, some categories may be those already learned by the image classification model, while others may be new categories predicted by the model. In other words, the image classification model has the ability to classify images with known categories, and also has the ability to classify images with unknown categories.
[0101] For a more detailed understanding of the image classification model training method provided in the embodiments of this application, please refer to [link / reference]. Figure 3 , Figure 3This is a flowchart illustrating the image classification model training method provided in this application embodiment. This image classification model training method can be applied to terminal devices such as mobile phones, computers, and tablets, or computer equipment such as servers. This image classification model training method can improve the image classification ability of the image classification model. The image classification model training method may include the following steps:
[0102] Step 210: Obtain the original image set, determine at least two views corresponding to each original image in the original image set, and determine the views as training images to obtain the training image set, which includes labeled training images and unlabeled training images.
[0103] Step 220: Obtain the image feature vector corresponding to each training image, and determine the contrastive learning loss corresponding to the image classification model based on the image feature vector;
[0104] Step 230: Input the image feature vector into the image classification model, output the image class probability corresponding to each training image, and determine the classification loss corresponding to the image classification model based on the image class probability and the labels of the labeled training images.
[0105] Step 240: Determine the target training image from the unlabeled training images based on the image category probability. The image category of the target training image is the known image category corresponding to the training image.
[0106] Step 250: Determine the local entropy regularization loss corresponding to the image classification model based on the image category probability corresponding to the target training image;
[0107] Step 260: Determine the model loss corresponding to the image classification model based on the contrastive learning loss, classification loss, and local entropy regularization loss, and update the model parameters of the image classification model according to the model loss to iteratively train the image classification model.
[0108] Steps 210 to 240 will be described in detail below.
[0109] In step 210, the original image set is obtained, at least two views corresponding to each original image in the original image set are determined, and the views are determined as training images to obtain the training image set, which includes labeled training images and unlabeled training images.
[0110] This process involves first acquiring an original image set, then performing image enhancement on each image in the original image set, such as adjusting parameters like brightness, color, and sharpness, to generate at least two corresponding views. In this application, for each original image, two different views of that original image can be generated, and then all the views corresponding to the original images are determined as training images, thus forming a training image set.
[0111] The training image set includes labeled and unlabeled training images. Labeled training images are those whose category is known and are labeled using that category. Unlabeled training images are those whose category is unknown and no label has been assigned to them.
[0112] Each labeled training image corresponds to a labeled training image set, denoted as D. l ={(x i ,y i )}∈X×Y l , where x i Represented as image i, y i Y represents the labeled image category of image i. l The known image categories in the labeled training image set.
[0113] The entire training image set contains both labeled and unlabeled training images, denoted as D. u ={(x i ,y i )}∈X×Y u , where Y u This indicates that the training image sets include both known image categories and unlabeled training image categories. In other words, within the entire training image set, the image categories of labeled training images are known, while unlabeled training images also have corresponding image categories, but those categories are unknown.
[0114] In step 220, the image feature vector corresponding to each training image is obtained, and the contrastive learning loss corresponding to the image classification model is determined based on the image feature vector.
[0115] The training images can be input into the trained image feature extraction model. The model then extracts the initial image features of each training image, which can be vectors. The trained image feature extraction model can be the DINO_VITB16 model, which extracts an image feature vector from a training image. This image feature vector can contain multiple sub-image feature vectors.
[0116] In this embodiment of the application, in the training image set, taking a certain training image as an example, if other training images are similar in image category to the training image, then the similar training images are positive samples for the training image. If other training images are not similar in image category to the training image, then the dissimilar training images are negative samples for the training image.
[0117] In this embodiment, the representation of data can be learned by comparing positive and negative samples, that is, by comparing different training images to learn the data's identifier. By minimizing the contrastive learning loss, the image classification model can learn to distinguish between similar and dissimilar samples, thereby improving the model's ability to classify images of different image categories.
[0118] Please see Figure 4 , Figure 4 This is a schematic diagram of the steps included in step 220 provided in the embodiments of this application.
[0119] In some implementations, the contrastive learning loss corresponding to the image classification model is determined based on the image feature vector, including the following steps:
[0120] Step 301: Input the image feature vector into the multilayer perceptron and output the target image feature vector corresponding to each training image;
[0121] Step 302: Calculate the unsupervised contrastive learning loss corresponding to the image classification model based on the target image feature vector and the first temperature coefficient;
[0122] Step 303: Determine the supervised contrastive learning loss corresponding to the image classification model based on the second temperature coefficient and the target image feature vector of the labeled training images;
[0123] Step 304: Determine the contrastive learning loss corresponding to the image classification model based on the unsupervised contrastive learning loss and the supervised contrastive learning loss.
[0124] Steps 301 to 304 will be described in detail below.
[0125] In step 301, the image feature vector is input into the multilayer perceptron, and the target image feature vector corresponding to each training image is output.
[0126] The Multi-layer Perceptron (MLP) can be understood as a neural network model capable of handling complex nonlinear relationships because its hidden layers can learn the nonlinear characteristics of the data. After inputting image feature vectors into the MLP, it outputs a target image feature vector corresponding to each training image. This target image feature vector can be understood as a feature vector extracted from the image feature vectors using depth, allowing for a more accurate representation of the corresponding training image.
[0127] In step 302, the unsupervised contrastive learning loss corresponding to the image classification model is calculated based on the target image feature vector and the first temperature coefficient.
[0128] In this application, the unsupervised contrastive learning loss corresponding to the image classification model can be determined based on the training images. Since the training image set contains many unlabeled training images whose categories are unknown, guiding the training of the image classification model with the unsupervised contrastive learning loss enables the trained model to accurately classify images of unknown categories.
[0129] In some implementations, the unsupervised contrastive learning loss corresponding to the image classification model is calculated based on the target image feature vector and the first temperature coefficient, including:
[0130] (1.1) Identify associated training images generated from the same original image in the training images;
[0131] (1.2) Determine the first distance between the associated training images based on the first temperature coefficient and the feature vector of the target image of the associated training images;
[0132] (1.3) Determine the second distance between any training image and an unrelated training image based on the first temperature coefficient and the target image feature vector of the training image;
[0133] (1.4) Determine the unsupervised contrastive learning loss corresponding to the image classification model based on the first distance and the second distance.
[0134] Please refer to the formula for unsupervised contrastive learning loss to understand this concept. The formula for unsupervised contrastive learning loss is as follows:
[0135]
[0136] in, The loss is unsupervised contrastive learning, where B represents a batch of training images in the current training process of the image classification model, i represents a training image, and n represents non-associated training images other than training image i. To associate the first distance between training images, The second distance between the training image and the unrelated training images is exp, which refers to an exponential function with the natural constant e as the base, τ. u The first temperature coefficient represents the unsupervised contrastive learning loss.
[0137] z i For training image x i The target image feature vector, z i ′ is the image x i The target image feature vector, z i =φ(f(x) i )), z i ′=φ(f(x i ′)), f is the image feature extraction model, φ is the multilayer perceptron, and T is the transpose matrix.
[0138] In this application, we first obtain associated training images generated based on the same original image, that is, training image x. i and training image x i Then, the training image x i and training image x i After image feature extraction is performed by the image feature extraction model, the training image x is obtained. i Image feature vector f(x) i ), to obtain the training image x i Image feature vector f(x) i Then, the image feature vector is input into a multilayer perceptron, and the training image x is output. i Target image feature vector z i Output training image x i The target image feature vector z' i ′.
[0139] Since the associated training images are generated based on an original image, they are quite similar. This allows the first distance between the associated training images to be as small as possible. Specifically, the first distance between the associated training images can be determined by the first temperature coefficient and the feature vector of the target image of the associated training image.
[0140] Since the training images in the associated training are different from each other, and the non-associated images can be understood as negative samples relative to the associated training images, the first distance between the associated training images and the non-associated images can be maximized to distinguish the training images of different image categories. Specifically, the second distance between any training image and the non-associated training image can be determined by the first temperature coefficient and the feature vector of the target image of the training image.
[0141] Subsequently, the unsupervised contrastive learning loss corresponding to the image classification model can be determined by inputting the first distance and the second distance into the formula of the unsupervised learning contrastive loss mentioned above.
[0142] In step 303, the supervised contrastive learning loss corresponding to the image classification model is determined based on the second temperature coefficient and the target image feature vector of the labeled training image.
[0143] In this application, the supervised contrastive learning loss corresponding to the image classification model can be determined based on the training images. The supervised contrastive learning loss utilizes labeled training images, enabling the image classification model to learn to classify or regress the label information of the training images, thereby allowing the image classification model to accurately classify images of known image categories.
[0144] In some implementations, the supervised contrastive learning loss corresponding to the image classification model is determined based on the second temperature coefficient and the target image feature vector of the labeled training images, including:
[0145] (2.1) Among the labeled training images, identify the training images with the same label;
[0146] (2.2) Determine the third distance between training images with the same label based on the second temperature coefficient and the feature vector of the target image of the training images with the same label;
[0147] (2.3) Based on the second temperature coefficient and the target image feature vector of the labeled training images, determine the fourth distance between any labeled training image and other labeled training images;
[0148] (2.4) Determine the supervised contrastive learning loss corresponding to the image classification model based on the third and fourth distances.
[0149] Please refer to the formula for supervised contrastive learning loss to understand this concept. The formula for supervised contrastive learning loss is as follows:
[0150]
[0151] in, For supervised contrastive learning loss, b l N represents the set of labeled training images with known image categories in the current training process of the image classification model. i Let i be a set of training images corresponding to each known image category, where i is a labeled training image, and q is the set of images corresponding to x. i The first image is a set of labeled training images with the same label information, and the second image is a set of labeled training images other than training image i. The third distance between labeled training images with the same label. Let τ be the fourth distance between any labeled training image and other labeled training images. T is the transpose matrix. c This is the second temperature coefficient. i Let z′ be the feature vector of the target image for a given labeled training image. q Let z′ be the target image feature vector of the labeled training image with the same label as the labeled training image. n This is the target image feature vector compared to other labeled training images besides this labeled training image.
[0152] Since labeled training images with the same label belong to the same image category, the distance between them can be minimized. Specifically, a third distance between training images with the same label can be determined using a second temperature coefficient and the feature vector of the target image of the training images with the same label.
[0153] Since the labeled training images belong to different image categories, the distance between labeled training images with different labels can be maximized. Specifically, a fourth distance between any labeled training image and other labeled training images can be determined based on the second temperature coefficient and the target image feature vector of the labeled training images.
[0154] Subsequently, the supervised contrastive learning loss corresponding to the image classification model can be determined by inputting the third and fourth distances into the formula of the supervised learning contrastive loss mentioned above.
[0155] In step 304, the contrastive learning loss corresponding to the image classification model is determined based on the unsupervised contrastive learning loss and the supervised contrastive learning loss.
[0156] After obtaining the supervised contrastive learning loss and the unsupervised contrastive learning loss, the final contrastive learning loss corresponding to the image classification model can be determined.
[0157] In some implementations, the weights corresponding to the supervised contrastive learning loss and the unsupervised contrastive learning loss can be obtained, and then the supervised contrastive learning loss and the unsupervised contrastive learning loss can be weighted and summed according to their weights to obtain the contrastive learning loss. The specific calculation formula is as follows:
[0158]
[0159] Among them, L rep Let λ be the contrastive learning loss corresponding to the calculated image classification model, where λ is the weight corresponding to the supervised contrastive learning loss and 1-λ is the weight corresponding to the unsupervised contrastive learning loss.
[0160] As can be seen from steps 301 to 304, by obtaining the supervised contrastive learning loss and unsupervised contrastive learning loss corresponding to the image classification model, the contrastive learning loss determined by the supervised contrastive learning loss and unsupervised contrastive learning loss can guide the image classification model to classify images of known image categories and images of unknown image categories during the training process, thereby improving the accuracy of image classification.
[0161] In step 230, the image feature vector is input into the image classification model, the image class probability corresponding to each training image is output, and the classification loss corresponding to the image classification model is determined based on the image class probability and the label of the labeled training image.
[0162] Each image feature vector can be input into an image classification model, which can then output the probability that the training image corresponding to that feature vector belongs to each predicted image category. The classification loss for the image classification model is then determined based on the image category probabilities and the labels of the labeled training images.
[0163] The classification loss of an image classification model helps improve its ability to recognize new image categories in training images.
[0164] Please refer to the following: Figure 5 , Figure 5 This is a flowchart illustrating the steps included in step 230 of the embodiments of this application.
[0165] In some implementations, the classification loss corresponding to the image classification model is determined based on the image class probability and the labels of the labeled training images, including:
[0166] Step 401: Determine the first cross-entropy loss corresponding to the image classification model based on the image category probabilities of the unlabeled training images;
[0167] Step 402: Determine the second cross-entropy loss corresponding to the image classification model based on the image category probabilities and labels of the labeled training images;
[0168] Step 403: Determine the average entropy regularization constraint corresponding to the image classification model based on the image category probabilities of the training images;
[0169] Step 404: Determine the classification loss corresponding to the image classification model based on the first cross-entropy loss, the second cross-entropy loss, and the average entropy regularization constraint.
[0170] Steps 401 to 404 will be described in detail below.
[0171] In step 401, the first cross-entropy loss corresponding to the image classification model is determined based on the image category probabilities of the unlabeled training images.
[0172] Unlabeled training images can be input into the formula for calculating the first cross-entropy loss to obtain the first cross-entropy loss corresponding to the image classification model. The formula for calculating the first cross-entropy loss is as follows:
[0173]
[0174] For unlabeled training image x i The corresponding image category probability is p i Unlabeled training image x i The probability of the image category corresponding to ' is q i ′, unlabeled training image x i and unlabeled training image x i ' represents two views generated based on the original image. Let B be the first cross-entropy loss, B be the number of training images in the current training process of the image classification model, and l be the image category of the unlabeled training images.
[0175] In step 402, the second cross-entropy loss corresponding to the image classification model is determined based on the image class probabilities and labels of the labeled training images.
[0176] The image class probabilities and true labels of the labeled training images can be input into the formula for calculating the second cross-entropy loss, thus obtaining the second cross-entropy loss corresponding to the image classification model. The formula for calculating the second cross-entropy loss is as follows:
[0177]
[0178] in Let B be the second cross-entropy loss, B be the number of training images in the current training process of the image classification model, and l be the image category of the labeled training image. For the labeled training image x... i The corresponding image category probability is p i , and y i For the labeled training image x i The corresponding tags.
[0179] In step 403, the average entropy regularization constraint corresponding to the image classification model is determined based on the image category probability of the training images.
[0180] Understandably, image classification models can output the probability of each predicted image class corresponding to the training image. Some predicted image classes have high probability of image class, while others have low probability of image class. This can lead to overfitting of the image classification model during training.
[0181] Therefore, in this application, by employing average entropy regularization, the distribution of image class probabilities for each predicted image class corresponding to the training image is made relatively uniform, avoiding situations where the probability of a training image in some predicted image classes is 0.9 while in others it is 0.01. If there are ten predicted image classes corresponding to the training image, then by using average entropy regularization, the probability of each predicted image class corresponding to the training image can be made close to 1 / 10.
[0182] The advantage of using average entropy regularization is that it can improve the generalization ability of image classification models, enabling them to recognize and classify images of more image categories.
[0183] In some implementations, the average entropy regularization constraint corresponding to the image classification model is determined based on the image class probabilities of the training images, including:
[0184] (1.1) Identify associated training images generated from the same original image in the training images;
[0185] (1.2) Determine the sum of the image category probabilities of the associated training images;
[0186] (1.3) Determine the number of training images corresponding to the image classification model in the current training process;
[0187] (1.4) Determine the average entropy regularization constraint corresponding to the image classification model based on the sum and quantity.
[0188] In this process, two views are generated based on the same original image; these two views are associated training images. For example, the image class probabilities corresponding to these two associated training images are p, respectively. i and p i We can obtain the sum of the two values and then input this sum into the formula for calculating the average image class probability, thus obtaining the average image class probability corresponding to each of the two associated training images. Similarly, for all associated training images in the training image set, we can obtain the average image class probability corresponding to each group of associated training images, and then add the average image class probabilities corresponding to each group of associated training images to obtain the average image class probability corresponding to the training image in the current training process. The formula for calculating the average image class probability is as follows:
[0189]
[0190] in, Let p be the average image class probability, B be the number of training images corresponding to the image classification model in the current training process, and p be the average image class probability. i and p i ′ represents the probability of the image category corresponding to each associated training image.
[0191] Then, the average image category probability is input into the entropy regularization calculation formula:
[0192]
[0193] in, is the average entropy regularization constraint, and x is the associated training image.
[0194] In step 404, the classification loss corresponding to the image classification model is determined based on the first cross-entropy loss, the second cross-entropy loss, and the average entropy regularization constraint.
[0195] The classification loss of the image classification model, determined by the first cross-entropy loss, the second cross-entropy loss, and the average entropy regularization constraint, helps to improve the image classification model's ability to recognize new image categories in the training images.
[0196] In some implementations, the classification loss corresponding to the image classification model is determined based on the first cross-entropy loss, the second cross-entropy loss, and the average entropy regularization constraint, including:
[0197] (2.1) Determine the weight coefficients corresponding to the average entropy regularization constraint, and multiply the weight coefficients by the average entropy regularization constraint to obtain the target average entropy regularization constraint.
[0198] (2.2) The classification loss corresponding to the training image is determined based on the first cross-entropy loss, the second cross-entropy loss and the target average entropy regularization constraint.
[0199] The formula for calculating the classification loss is as follows:
[0200]
[0201] L cls For classifying losses, For average entropy regularization constraints, For the first cross-entropy loss, The second cross-entropy loss is given by ε, where ε is the weighting coefficient corresponding to the average entropy regularization constraint. The target is the average entropy regularization constraint.
[0202] The first calculation result is obtained by subtracting the target average entropy regularization constraint from the first cross-entropy loss, and then multiplying it by the corresponding weight coefficient 1-λ. The second calculation result is obtained by multiplying the second cross-entropy loss by the corresponding weight coefficient λ. Finally, the first and second calculation results are added together to obtain the classification loss corresponding to the image classification model.
[0203] In some implementations, the classification loss corresponding to the training image is determined based on the first cross-entropy loss, the second cross-entropy loss, and the target average entropy regularization constraint, including:
[0204] (2.2.1) Determine the two-view divergence constraint corresponding to the associated training images based on the quantity and the image category probability of the associated training images;
[0205] (2.2.2) The classification loss corresponding to the training image is determined based on the first cross-entropy loss, the second cross-entropy loss, the target average entropy regularization constraint, and the two-view divergence constraint.
[0206] In order to maintain the consistency of predictions from the image classification model to two associated training images generated from a given original image, and to identify the potential training images corresponding to the known image categories in the training images, the two-view divergence constraint corresponding to the associated training images can be calculated. The specific calculation formula is as follows:
[0207]
[0208] Among them, D KL (p i ||p i ′) represents the two-view divergence constraint (two-view KL divergence constraint), B is the number of training images, and x i and x i ' represents the associated training image, p(x) i ) and p(x i ′) represent the image category probabilities of the associated training images.
[0209] Then, the two-view divergence constraint is input into the classification loss update formula to update the classification loss. The specific formula is as follows:
[0210]
[0211] Among them, L cls This is the classification loss that will ultimately be used in the image classification model.
[0212] In this way, the two-view divergence constraint guides the model to generate similar predictive distributions for samples from two views of the same image. This helps avoid mismatches between the image categories corresponding to the associated training images and simultaneously generates training images with more reliable potential known image categories. This improves the model's understanding and utilization of unlabeled training images, thereby enhancing the image classification model's ability to identify new image categories.
[0213] As can be seen from steps 401 to 404, in this embodiment of the application, the classification loss calculated by the first cross-entropy loss, the second cross-entropy loss and the target average entropy regularization constraint helps to prevent overfitting of the image classification model during the training process and helps to improve the image classification model's ability to recognize new image categories in the training images.
[0214] In step 240, the target training image is determined from the unlabeled training images based on the image category probability, and the image category of the target training image is the known image category corresponding to the training image.
[0215] Unlabeled training images are identified from the training images. Then, the image category probability with the highest value corresponding to the unlabeled training image is determined, and the image category corresponding to this probability is designated as the category of the unlabeled training image. Next, it is determined whether the image category corresponding to each unlabeled training image is a known image category corresponding to the training image. If the image category corresponding to an unlabeled training image is a known image category corresponding to the training image, then the unlabeled training image is designated as the target training image.
[0216] In step 250, the local entropy regularization loss corresponding to the image classification model is determined based on the image category probability corresponding to the target training image.
[0217] In this application, the local entropy regularization loss of the image classification model is determined by the image category probability corresponding to the target training image. This local entropy regularization constraint aims to optimize the distribution of unlabeled training images with potential known image categories in the unlabeled training samples. This helps the image classification model retain knowledge related to known image categories when learning new image categories, thereby avoiding the problem of forgetting knowledge related to known image categories during the training process. This method is similar to a memory retention mechanism, helping the image classification model maintain its awareness of known image categories when learning new image categories.
[0218] Please see Figure 6 , Figure 6 This is a flowchart illustrating the steps included in step 250 of the embodiments of this application.
[0219] In some implementations, the local entropy regularization loss corresponding to the image classification model is determined based on the image class probability corresponding to the target training image, including:
[0220] Step 501: Determine the entropy regularization constraint corresponding to the image classification model based on the third temperature coefficient and the image category probability of the target training image;
[0221] Step 502: Determine the number of training images corresponding to the image classification model in the current training process;
[0222] Step 503: Determine the local entropy regularization loss corresponding to the image classification model based on the entropy regularization constraint corresponding to the target training image and the number of targets.
[0223] Steps 501 to 503 will be described in detail below.
[0224] In step 501, the entropy regularization constraint corresponding to the image classification model is determined based on the third temperature coefficient and the image category probability of the target training image.
[0225] Specifically, the entropy regularization constraint corresponding to the image classification model can be determined by calculating the average prediction coefficient and then using the average prediction coefficient, the third temperature coefficient, and the image category probability of the target training image.
[0226] The specific calculation formula is as follows:
[0227]
[0228] Where H is entropy regularization, x i For the target training image, p(x) i ) represents the image class probability of the target training image, τ o The third temperature coefficient, Δ j This represents the average prediction coefficient.
[0229] The formula for calculating the average predictive coefficient is as follows:
[0230]
[0231] Where λ ler is the weight coefficient, which can be 0.4, and j is the set of known image categories corresponding to all target training images. The average predicted probability for each known image category. For example, if image category 1 corresponds to three target training images, then... It is the mean of the image class probabilities of the three target training images.
[0232] In step 502, the number of training images corresponding to the image classification model in the current training process is determined.
[0233] In other words, it refers to the number of training images in the current batch during the current training process of the image classification model.
[0234] In step 503, the local entropy regularization loss corresponding to the image classification model is determined based on the entropy regularization constraint corresponding to the target training image and the number of targets.
[0235] Specifically, for all target training images, the entropy regularization memory corresponding to each target training image can be accumulated, and then divided by the number of training images. The specific calculation method is as follows:
[0236]
[0237] Among them, L LER This is the loss due to local entropy regularization. i =1 indicates that it is the target training image.
[0238] As shown in steps 501 to 503, target training images whose image categories are known are selected from the unlabeled training images, and then the corresponding local entropy regularization loss is determined. This helps the image classification model retain knowledge related to known image categories when learning new image categories, thereby avoiding the problem of forgetting knowledge related to known image categories during the training process.
[0239] In step 260, the model loss corresponding to the image classification model is determined based on the contrastive learning loss, classification loss, and local entropy regularization loss, and the model parameters of the image classification model are updated based on the model loss to iteratively train the image classification model.
[0240] By combining contrastive learning loss, classification loss, and local entropy regularization loss, the model loss corresponding to the image classification model is determined. The model loss calculated in this way is more accurate, and the model parameters of the image classification model can be adjusted more accurately through the model loss to achieve iterative training of the image classification model. The final trained image classification model will have more accurate image classification.
[0241] In some implementations, the model loss corresponding to the image classification model is determined based on contrastive learning loss, classification loss, and local entropy regularization loss, including:
[0242] (1.1) Multiply the sum of the contrastive learning loss and the classification loss by the weight of the first loss to obtain the first result;
[0243] (1.2) Multiply the local entropy regularization loss by the second loss weight to obtain the second result;
[0244] (1.3) Add the first result and the second result together to obtain the model loss corresponding to the image classification model.
[0245] The specific calculation formula is as follows:
[0246] L=α·(L rep +L cls )+β·L LER .
[0247] L represents the model loss, α represents the weight of the first loss, and β represents the weight of the second loss. rep To compare learning loss, L cls For classification loss, L LER This represents the loss due to local entropy regularization.
[0248] After obtaining the model loss, the model parameters of the image classification model can be updated based on the model loss to iterate the model. The iterated model is then trained again, and the model loss for the next training iteration is obtained. The process is repeated, updating the model parameters based on the model loss. This allows for continuous iteration of the image classification model. Since the model loss calculated in each iteration is accurate, the model can be trained in the correct direction each time, thus improving training efficiency. The trained image classification model can classify and recognize images of known categories as well as those of unknown categories, achieving high accuracy in image classification.
[0249] As described in steps 210 to 260, in this embodiment, by acquiring an original image set, at least two views corresponding to each original image in the original image set are determined, and these views are used as training images to obtain a training image set, which includes labeled training images and unlabeled training images. The image feature vector corresponding to each training image is acquired, and the contrastive learning loss corresponding to the image classification model is determined based on the image feature vector. The image feature vector is input into the image classification model, and the image class probability corresponding to each training image is output. The classification loss corresponding to the image classification model is determined based on the image class probability and the label of the labeled training image. The target training image is determined from the unlabeled training images based on the image class probability, and the image class of the target training image is the known image class corresponding to the training image. The local entropy regularization loss corresponding to the image classification model is determined based on the image class probability corresponding to the target training image. The model loss corresponding to the image classification model is determined based on the contrastive learning loss, the classification loss, and the local entropy regularization loss. The model parameters of the image classification model are updated based on the model loss to iteratively train the image classification model. In this embodiment, by determining the contrastive learning loss corresponding to the image classification model, the image classification model's ability to extract image features from different training images can be improved, as can its classification ability. By determining the image category probability of the target training image with a known image category in unlabeled training images, the local entropy regularization loss corresponding to the image classification model is determined. This helps the image classification model retain knowledge related to known image categories when learning new image categories, avoiding catastrophic forgetting during training. Furthermore, determining the classification loss corresponding to the training images helps improve the image classification model's ability to recognize new image categories in the training images. Therefore, the model loss determined by the contrastive learning loss, classification loss, and local entropy regularization loss can more accurately guide the training direction of the image classification model, thereby avoiding catastrophic forgetting during training and resulting in a higher image classification accuracy after training.
[0250] Please refer to the following: Figure 7 , Figure 7 This is another schematic flowchart illustrating the image classification model training method provided in this application embodiment. The image classification model training method may include the following steps:
[0251] Step 601: Obtain the original image set, determine at least two views corresponding to each original image in the original image set, and use the views as training images to obtain the training image set;
[0252] Step 602: Input the image feature vector into the multilayer perceptron and output the target image feature vector corresponding to each training image;
[0253] Step 603: Calculate the unsupervised contrastive learning loss corresponding to the image classification model based on the target image feature vector and the first temperature coefficient;
[0254] Step 604: Determine the supervised contrastive learning loss corresponding to the image classification model based on the second temperature coefficient and the target image feature vector of the labeled training images;
[0255] Step 605: Determine the contrastive learning loss corresponding to the image classification model based on the unsupervised contrastive learning loss and the supervised contrastive learning loss;
[0256] Step 606: Input the image feature vector into the image classification model and output the image class probability corresponding to each training image;
[0257] Step 607: Determine the first cross-entropy loss corresponding to the image classification model based on the image category probabilities of the unlabeled training images;
[0258] Step 608: Determine the second cross-entropy loss corresponding to the image classification model based on the image class probabilities and labels of the labeled training images;
[0259] Step 609: Determine the average entropy regularization constraint corresponding to the image classification model based on the image category probabilities of the training images;
[0260] Step 610: Determine the target training image from the unlabeled training images based on the image category probability. The image category of the target training image is the known image category corresponding to the training image.
[0261] Step 611: Determine the entropy regularization constraint corresponding to the image classification model based on the third temperature coefficient and the image category probability of the target training image;
[0262] Step 612: Determine the number of training images corresponding to the image classification model in the current training process, and determine the local entropy regularization loss corresponding to the image classification model based on the entropy regularization constraint corresponding to the target training image and the number of targets.
[0263] Step 613: Determine the model loss corresponding to the image classification model based on the contrastive learning loss, classification loss, and local entropy regularization loss. Update the model parameters of the image classification model based on the model loss to iteratively train the image classification model.
[0264] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed description of the image classification model training method above, which will not be repeated here.
[0265] In this embodiment, by determining the contrastive learning loss corresponding to the image classification model, the image classification model's ability to extract image features from different training images can be improved, as can its classification ability. By determining the image category probability of the target training image with a known image category in unlabeled training images, the local entropy regularization loss corresponding to the image classification model is determined. This helps the image classification model retain knowledge related to known image categories when learning new image categories, avoiding catastrophic forgetting during training. Furthermore, determining the classification loss corresponding to the training images helps improve the image classification model's ability to recognize new image categories in the training images. Therefore, the model loss determined by the contrastive learning loss, classification loss, and local entropy regularization loss can more accurately guide the training direction of the image classification model, thereby avoiding catastrophic forgetting during training and resulting in a higher image classification accuracy after training.
[0266] Please see Figure 8 , Figure 8 This is a schematic diagram of the image classification model training device provided in an embodiment of this application. This image classification model training device can be used to execute the image classification model training method described in the above embodiments.
[0267] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.
[0268] like Figure 8 As shown, the image classification model training device 700 includes:
[0269] The acquisition module 710 is used to acquire the original image set, determine at least two views corresponding to each original image in the original image set, and determine the views as training images to obtain the training image set, which includes labeled training images and unlabeled training images.
[0270] The first loss module 720 is used to obtain the image feature vector corresponding to each training image and determine the contrastive learning loss corresponding to the image classification model based on the image feature vector.
[0271] The second loss module 730 is used to input the image feature vector into the image classification model, output the image class probability corresponding to each training image, and determine the classification loss corresponding to the image classification model based on the image class probability and the label of the labeled training image.
[0272] The determination module 740 is used to determine the target training image from the unlabeled training images based on the image category probability, wherein the image category of the target training image is the known image category corresponding to the training image;
[0273] The third loss module 750 is used to determine the local entropy regularization loss corresponding to the image classification model based on the image category probability corresponding to the target training image.
[0274] The fourth loss module 760 is used to determine the model loss corresponding to the image classification model based on the contrastive learning loss, classification loss and local entropy regularization loss, and update the model parameters of the image classification model according to the model loss, so as to iteratively train the image classification model.
[0275] In some implementations, the first loss module 720 includes a feature extraction submodule, an unsupervised submodule, a supervised submodule, and a determination submodule;
[0276] The feature extraction submodule is used to input the image feature vector into the multilayer perceptron and output the target image feature vector corresponding to each training image;
[0277] The unsupervised submodule is used to calculate the unsupervised contrastive learning loss corresponding to the image classification model based on the feature vector of the target image and the first temperature coefficient.
[0278] The supervised submodule is used to determine the supervised contrastive learning loss corresponding to the image classification model based on the second temperature coefficient and the feature vector of the target image of the labeled training image.
[0279] The determination submodule is used to determine the contrastive learning loss corresponding to the image classification model based on the unsupervised contrastive learning loss and the supervised contrastive learning loss.
[0280] In some implementations, an unsupervised submodule is used to identify associated training images generated based on the same original image in the training images;
[0281] The first distance between the associated training images is determined based on the first temperature coefficient and the feature vector of the target image of the associated training images;
[0282] Based on the first temperature coefficient and the feature vector of the target image of the training image, a second distance between any training image and an unrelated training image is determined;
[0283] The unsupervised contrastive learning loss corresponding to the image classification model is determined based on the first distance and the second distance.
[0284] In some implementations, there is a supervisory submodule for identifying training images with the same label among the labeled training images;
[0285] Based on the second temperature coefficient and the feature vector of the target image of the training image with the same label, the third distance between the training images with the same label is determined.
[0286] Based on the second temperature coefficient and the target image feature vector of the labeled training images, a fourth distance between any labeled training image and other labeled training images is determined.
[0287] The supervised contrastive learning loss corresponding to the image classification model is determined based on the third and fourth distances.
[0288] In some implementations, the second loss module 730 includes a first sub-loss module, a second sub-loss module, a constraint sub-module, and a third sub-loss module;
[0289] The first sub-loss module is used to determine the first cross-entropy loss corresponding to the image classification model based on the image category probability of the unlabeled training images.
[0290] The second sub-loss module is used to determine the second cross-entropy loss corresponding to the image classification model based on the image class probabilities and labels of the labeled training images.
[0291] The constraint submodule is used to determine the average entropy regularization constraint corresponding to the image classification model based on the image category probability of the training images.
[0292] The third sub-loss module is used to determine the classification loss corresponding to the image classification model based on the first cross-entropy loss, the second cross-entropy loss, and the average entropy regularization constraint.
[0293] In some implementations, a constraint submodule is used to identify associated training images generated based on the same original image in the training images;
[0294] Determine the sum of the probabilities of image categories associated with the training images;
[0295] Determine the number of training images corresponding to the image classification model in the current training process;
[0296] The average entropy regularization constraint corresponding to the image classification model is determined based on the sum and quantity.
[0297] In some implementations, the third sub-loss module is used to determine the weight coefficients corresponding to the average entropy regularization constraint, and multiply the weight coefficients by the average entropy regularization constraint to obtain the target average entropy regularization constraint.
[0298] The classification loss corresponding to the training image is determined based on the first cross-entropy loss, the second cross-entropy loss, and the target average entropy regularization constraint.
[0299] In some implementations, the third sub-loss module is used to determine the two-view divergence constraint corresponding to the associated training image based on the quantity and the image class probability of the associated training image;
[0300] The classification loss corresponding to the training image is determined based on the first cross-entropy loss, the second cross-entropy loss, the target average entropy regularization constraint, and the two-view divergence constraint.
[0301] In some implementations, the determining module 740 is used to determine candidate training images with a higher probability than a preset threshold among unlabeled training images based on the image category probability.
[0302] The target image category of the candidate training images is determined based on the image category probability of the candidate training images;
[0303] Candidate training images whose target image category is a known image category are selected as target training images.
[0304] The third loss module 750 is used to determine the entropy regularization constraint corresponding to the image classification model based on the third temperature coefficient and the image category probability of the target training image.
[0305] Determine the number of training images corresponding to the image classification model in the current training process;
[0306] The local entropy regularization loss of the image classification model is determined based on the entropy regularization constraint corresponding to the target training image and the number of targets.
[0307] The fourth loss module 760 is used to multiply the sum of the contrastive learning loss and the classification loss by the first loss weight to obtain the first result;
[0308] Multiplying the local entropy regularization loss by the second loss weight yields the second result.
[0309] Add the first result and the second result to obtain the model loss corresponding to the image classification model.
[0310] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed description of the image classification model training method above, which will not be repeated here.
[0311] As can be seen from the listed company, in this embodiment of the application, the acquisition module 710 acquires the original image set, determines at least two views corresponding to each original image in the original image set, and determines the views as training images to obtain a training image set, which includes labeled training images and unlabeled training images; the first loss module 720 acquires the image feature vector corresponding to each training image, and determines the contrastive learning loss corresponding to the image classification model based on the image feature vector; the second loss module 730 inputs the image feature vector into the image classification model, outputs the image class probability corresponding to each training image, and determines the contrastive learning loss corresponding to the image classification model based on the image class probability and the labeled training images. The image classification model is iteratively trained by: labeling training images and determining the classification loss; determining the target training image from unlabeled training images based on image category probabilities, where the target training image's category is the known category corresponding to the training images; determining the local entropy regularization loss based on the target training image's category probabilities; and determining the model loss based on the contrastive learning loss, classification loss, and local entropy regularization loss, and updating the model parameters accordingly. In this embodiment, determining the contrastive learning loss improves the image classification model's ability to extract image features from different training images and enhances its classification capabilities. Determining the local entropy regularization loss based on the image category probabilities of the target training image (with known category among unlabeled training images) helps the model retain knowledge related to known image categories when learning new categories, preventing catastrophic forgetting during training. Furthermore, determining the classification loss corresponding to the training images helps improve the image classification model's ability to identify new image categories within the training images. Thus, the model loss determined by comparing the learning loss, classification loss, and local entropy regularization loss can more accurately guide the training direction of the image classification model, thereby avoiding catastrophic forgetting during training and resulting in a more accurate image classification model after training.
[0312] This application also provides a computer device, which can be a terminal, such as... Figure 9 As shown, it illustrates the structural diagram of the terminal involved in the embodiments of this application, specifically:
[0313] The computer device may include a radio frequency (RF) circuit 801, a memory 802 including one or more computer-readable storage media, an input unit 803, a display unit 804, a sensor 805, an audio circuit 806, a wireless fidelity (WiFi) module 807, a processor 808 including one or more processing cores, and a power supply 809, etc. Those skilled in the art will understand that... Figure 9 The terminal structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:
[0314] RF circuit 801 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and hands it over to one or more processors 808 for processing; additionally, it transmits uplink data to the base station. Typically, RF circuit 801 includes, but is not limited to, an antenna, at least one amplifier, a tuner, one or more oscillators, a Subscriber Identity Module (SIM) card, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, RF circuit 801 can also communicate wirelessly with networks and other devices. Wireless communication can use any communication standard or protocol, including but not limited to GSM, GPRS, CDMA, WCDMA, LTE, email, and SMS.
[0315] The memory 802 can be used to store software programs and modules. The processor 808 executes various functional applications and information retrieval by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the terminal (such as audio data, phone book, etc.). In addition, the memory 802 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 802 may also include a memory controller to provide access to the memory 802 for the processor 808 and the input unit 803.
[0316] Input unit 803 can be used to receive input digital or character information, and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to object settings and function control. Specifically, in one embodiment, input unit 803 may include a touch-sensitive surface and other input devices. A touch-sensitive surface, also known as a touch display or touchpad, can collect touch operations on or near the object (e.g., operations performed by the object using a finger, stylus, or any suitable object or accessory on or near the touch-sensitive surface), and drive corresponding connection devices according to a pre-set program. Optionally, the touch-sensitive surface may include a touch detection device and a touch controller. The touch detection device detects the touch orientation of the object and the signal generated by the touch operation, transmitting the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to the processor 808, and can receive and execute commands from the processor 808. Furthermore, various types of touch-sensitive surfaces, such as resistive, capacitive, infrared, and surface acoustic wave, can be used. In addition to the touch-sensitive surface, input unit 803 may also include other input devices. Specifically, other input devices may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.
[0317] The display unit 804 can be used to display information input by an object or information provided to an object, as well as various graphical object interfaces of the terminal. These graphical object interfaces can be composed of graphics, text, icons, video, and any combination thereof. The display unit 804 may include a display panel, optionally configured as a liquid crystal display (LCD), organic light-emitting diode (OLED), or similar form. Furthermore, a touch-sensitive surface may cover the display panel. When the touch-sensitive surface detects a touch operation on or near it, it transmits the information to the processor 808 to determine the type of touch event. Subsequently, the processor 808 provides corresponding visual output on the display panel according to the type of touch event.
[0318] The terminal may also include at least one sensor 805, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel according to the ambient light level, and the proximity sensor can turn off the display panel and / or backlight when the terminal is moved to the ear. As a type of motion sensor, a gravity acceleration sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, tapping), etc. Other sensors that the terminal may also be equipped with, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.
[0319] Audio circuitry 806, a speaker, and a microphone provide an audio interface between the device and the terminal. Audio circuitry 806 converts received audio data into electrical signals, transmits them to the speaker, and the speaker converts them into sound signals for output. Conversely, the microphone converts collected sound signals into electrical signals, which are then received by audio circuitry 806, converted back into audio data, and processed by processor 808. The processed data is then transmitted via RF circuitry 801 to, for example, another terminal, or output to memory 802 for further processing. Audio circuitry 806 may also include an earphone jack to facilitate communication between a peripheral headset and the terminal.
[0320] WiFi is a short-range wireless transmission technology. A terminal using a WiFi module 807 can help users send and receive emails, browse web pages, and access streaming media, providing wireless broadband internet access. Although Figure 9 WiFi module 807 is shown, but it is understood that it is not a necessary component of the terminal and can be omitted as needed without changing the essence of the invention.
[0321] The processor 808 is the control center of the terminal, connecting various parts of the phone via various interfaces and lines. It executes software programs and / or modules stored in the memory 802, and calls data stored in the memory 802, to perform various functions and process data, thereby providing overall monitoring of the phone. Optionally, the processor 808 may include one or more processing cores; preferably, the processor 808 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 808.
[0322] The terminal also includes a power supply 809 (such as a battery) to power various components. Preferably, the power supply can be logically connected to the processor 808 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 809 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.
[0323] Although not shown, the terminal may also include a camera, Bluetooth module, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 808 in the terminal loads the executable files corresponding to the processes of one or more applications into the memory 802 according to the following instructions, and the processor 808 runs the applications stored in the memory 802 to realize various functions:
[0324] Obtain the original image set, determine at least two views corresponding to each original image in the original image set, and use the views as training images to obtain the training image set, which contains labeled training images and unlabeled training images;
[0325] Obtain the image feature vector corresponding to each training image, and determine the contrastive learning loss corresponding to the image classification model based on the image feature vector;
[0326] The image feature vector is input into the image classification model, which outputs the image class probability corresponding to each training image. Based on the image class probability and the labels of the labeled training images, the classification loss corresponding to the image classification model is determined.
[0327] The target training image is determined from the unlabeled training images based on the image category probability. The image category of the target training image is the known image category corresponding to the training image.
[0328] Based on the image category probabilities corresponding to the target training image, the local entropy regularization loss corresponding to the image classification model is determined;
[0329] The model loss corresponding to the image classification model is determined by contrastive learning loss, classification loss and local entropy regularization loss, and the model parameters of the image classification model are updated according to the model loss to iteratively train the image classification model.
[0330] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed description of the image classification model training method above, which will not be repeated here.
[0331] In this embodiment, by acquiring an original image set, at least two views corresponding to each original image in the original image set are determined, and these views are used as training images to obtain a training image set, which includes labeled training images and unlabeled training images. The image feature vector corresponding to each training image is obtained, and the contrastive learning loss corresponding to the image classification model is determined based on the image feature vector. The image feature vector is input into the image classification model, and the image class probability corresponding to each training image is output. The classification loss corresponding to the image classification model is determined based on the image class probability and the labels of the labeled training images. The target training image is determined from the unlabeled training images based on the image class probability, and the image class of the target training image is the known image class corresponding to the training image. The local entropy regularization loss corresponding to the image classification model is determined based on the image class probability corresponding to the target training image. The model loss corresponding to the image classification model is determined based on the contrastive learning loss, the classification loss, and the local entropy regularization loss. The model parameters of the image classification model are updated based on the model loss to iteratively train the image classification model. In this embodiment, by determining the contrastive learning loss corresponding to the image classification model, the image classification model's ability to extract image features from different training images can be improved, as can its classification ability. By determining the image category probability of the target training image with a known image category in unlabeled training images, the local entropy regularization loss corresponding to the image classification model is determined. This helps the image classification model retain knowledge related to known image categories when learning new image categories, avoiding catastrophic forgetting during training. Furthermore, determining the classification loss corresponding to the training images helps improve the image classification model's ability to recognize new image categories in the training images. Therefore, the model loss determined by the contrastive learning loss, classification loss, and local entropy regularization loss can more accurately guide the training direction of the image classification model, thereby avoiding catastrophic forgetting during training and resulting in a higher image classification accuracy after training.
[0332] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.
[0333] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the image classification model training methods provided in embodiments of this application. For example, the instructions can execute the following steps:
[0334] Obtain the original image set, determine at least two views corresponding to each original image in the original image set, and use the views as training images to obtain the training image set, which contains labeled training images and unlabeled training images;
[0335] Obtain the image feature vector corresponding to each training image, and determine the contrastive learning loss corresponding to the image classification model based on the image feature vector;
[0336] The image feature vector is input into the image classification model, which outputs the image class probability corresponding to each training image. Based on the image class probability and the labels of the labeled training images, the classification loss corresponding to the image classification model is determined.
[0337] The target training image is determined from the unlabeled training images based on the image category probability. The image category of the target training image is the known image category corresponding to the training image.
[0338] Based on the image category probabilities corresponding to the target training image, the local entropy regularization loss corresponding to the image classification model is determined;
[0339] The model loss corresponding to the image classification model is determined by contrastive learning loss, classification loss and local entropy regularization loss, and the model parameters of the image classification model are updated according to the model loss to iteratively train the image classification model.
[0340] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed description of the image classification model training method above, which will not be repeated here.
[0341] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the image classification model training method provided in the various optional implementations of the above embodiments.
[0342] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.
[0343] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0344] Since the instructions stored in the computer-readable storage medium can execute the steps in any of the image classification model training methods provided in the embodiments of this application, the beneficial effects that any of the image classification model training methods provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.
[0345] The above provides a detailed description of an image classification model training method, apparatus, device, and medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for training an image classification model, characterized in that, include: Obtain the original image set, determine at least two views corresponding to each original image in the original image set, and determine the views as training images to obtain the training image set, which includes labeled training images and unlabeled training images; Obtain the image feature vector corresponding to each training image, and determine the contrastive learning loss corresponding to the image classification model based on the image feature vector; The image feature vector is input into the image classification model, the image category probability corresponding to each training image is output, and the classification loss corresponding to the image classification model is determined based on the image category probability and the label of the labeled training image. The target training image is determined from the unlabeled training images based on the image category probability, and the image category of the target training image is the known image category corresponding to the training image. Based on the image category probability corresponding to the target training image, the local entropy regularization loss corresponding to the image classification model is determined; The model loss corresponding to the image classification model is determined based on the contrastive learning loss, the classification loss, and the local entropy regularization loss, and the model parameters of the image classification model are updated based on the model loss to iteratively train the image classification model.
2. The image classification model training method according to claim 1, characterized in that, The step of determining the contrastive learning loss corresponding to the image classification model based on the image feature vector includes: The image feature vector is input into a multilayer perceptron, and the target image feature vector corresponding to each training image is output. The unsupervised contrastive learning loss corresponding to the image classification model is calculated based on the target image feature vector and the first temperature coefficient. The supervised contrastive learning loss corresponding to the image classification model is determined based on the second temperature coefficient and the feature vector of the target image of the labeled training image; Based on the unsupervised contrastive learning loss and the supervised contrastive learning loss, the contrastive learning loss corresponding to the image classification model is determined.
3. The image classification model training method according to claim 2, characterized in that, The step of calculating the unsupervised contrastive learning loss corresponding to the image classification model based on the target image feature vector and the first temperature coefficient includes: Identify associated training images generated from the same original image from the training images; Based on the first temperature coefficient and the target image feature vector of the associated training images, a first distance between the associated training images is determined; Based on the first temperature coefficient and the target image feature vector of the training images, a second distance between any training image and other training images is determined; The unsupervised contrastive learning loss corresponding to the image classification model is determined based on the first distance and the second distance.
4. The image classification model training method according to claim 2, characterized in that, The step of determining the supervised contrastive learning loss corresponding to the image classification model based on the second temperature coefficient and the feature vector of the target image of the labeled training image includes: Among the labeled training images, the training images with the same label are identified; A third distance is determined between the training images with the same label based on the second temperature coefficient and the target image feature vector of the training images with the same label. Based on the second temperature coefficient and the target image feature vector of the labeled training images, a fourth distance between any of the labeled training images and other labeled training images is determined; The supervised contrastive learning loss corresponding to the image classification model is determined based on the third distance and the fourth distance.
5. The image classification model training method according to claim 1, characterized in that, The step of determining the classification loss corresponding to the image classification model based on the image category probability and the labels of the labeled training images includes: Based on the image category probabilities of the unlabeled training images, the first cross-entropy loss corresponding to the image classification model is determined; Based on the image category probabilities of the labeled training images and the labels of the labeled training images, the second cross-entropy loss corresponding to the image classification model is determined; The average entropy regularization constraint corresponding to the image classification model is determined based on the image category probability of the training images. The classification loss corresponding to the image classification model is determined based on the first cross-entropy loss, the second cross-entropy loss, and the average entropy regularization constraint.
6. The image classification model training method according to claim 5, characterized in that, The determination of the average entropy regularization constraint corresponding to the image classification model based on the image category probabilities of the training images includes: Identify associated training images generated from the same original image from the training images; Determine the sum of the image category probabilities of the associated training images; Determine the number of training images corresponding to the image classification model in the current training process; The average entropy regularization constraint corresponding to the image classification model is determined based on the sum and the quantity.
7. The image classification model training method according to claim 6, characterized in that, The step of determining the classification loss corresponding to the image classification model based on the first cross-entropy loss, the second cross-entropy loss, and the average entropy regularization constraint includes: Determine the weight coefficients corresponding to the average entropy regularization constraint, and multiply the weight coefficients by the average entropy regularization constraint to obtain the target average entropy regularization constraint; The classification loss corresponding to the training image is determined based on the first cross-entropy loss, the second cross-entropy loss, and the target average entropy regularization constraint.
8. The image classification model training method according to claim 7, characterized in that, The step of determining the classification loss corresponding to the training image based on the first cross-entropy loss, the second cross-entropy loss, and the target average entropy regularization constraint includes: Based on the quantity and the image category probability of the associated training image, the dual-view divergence constraint corresponding to the associated training image is determined; The classification loss corresponding to the training image is determined based on the first cross-entropy loss, the second cross-entropy loss, the target average entropy regularization constraint, and the two-view divergence constraint.
9. The image classification model training method according to claim 1, characterized in that, The step of determining the target training image from the unlabeled training images based on the image category probability includes: Candidate training images with a probability of image category higher than a preset threshold are identified from the unlabeled training images; The target image category of the candidate training image is determined based on the image category probability of the candidate training image; The candidate training images whose target image category is the known image category are determined as the target training images.
10. The image classification model training method according to claim 1, characterized in that, The step of determining the local entropy regularization loss corresponding to the image classification model based on the image category probability corresponding to the target training image includes: The entropy regularization constraint corresponding to the image classification model is determined based on the third temperature coefficient and the image category probability of the target training image. Determine the number of training images corresponding to the image classification model in the current training process; The local entropy regularization loss corresponding to the image classification model is determined based on the entropy regularization constraint corresponding to the target training image and the number of targets.
11. The image classification model training method according to claim 1, characterized in that, The step of determining the model loss corresponding to the image classification model based on the contrastive learning loss, the classification loss, and the local entropy regularization loss includes: The sum of the contrastive learning loss and the classification loss is multiplied by the first loss weight to obtain the first result; Multiply the local entropy regularization loss by the second loss weight to obtain the second result; The first result and the second result are added together to obtain the model loss corresponding to the image classification model.
12. An image classification model training device, characterized in that, include: The acquisition module is used to acquire the original image set, determine at least two views corresponding to each original image in the original image set, and determine the views as training images to obtain the training image set, which includes labeled training images and unlabeled training images. The first loss module is used to obtain the image feature vector corresponding to each training image and determine the contrastive learning loss corresponding to the image classification model based on the image feature vector. The second loss module is used to input the image feature vector into the image classification model, output the image category probability corresponding to each training image, and determine the classification loss corresponding to the image classification model based on the image category probability and the label of the labeled training image. The determination module is used to determine the target training image from the unlabeled training images according to the image category probability, wherein the image category of the target training image is the known image category corresponding to the training image; The third loss module is used to determine the local entropy regularization loss corresponding to the image classification model based on the image category probability corresponding to the target training image. The fourth loss module is used to determine the model loss corresponding to the image classification model based on the contrastive learning loss, the classification loss, and the local entropy regularization loss, and to update the model parameters of the image classification model based on the model loss, so as to iteratively train the image classification model.
13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to execute the image classification model training method according to any one of claims 1 to 11.
14. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the image classification model training method according to any one of claims 1 to 11.
