Image classification method and system based on multi-model joint contrastive learning

Abstract

The present invention proposes an image classification method and system based on multi-model joint contrast learning, including: the present invention proposes a multi-model joint contrast learning algorithm to guide different models to interact at the feature level, and mutually transfer the respective contrast similarity probability distributions between multiple models. The use of contrast learning can make samples of the same category close in the feature space, and samples of different categories far away in the feature space, thereby obtaining a discriminative representation space. In the scenario of multi-model joint learning, contrast learning can establish a relationship between the feature vectors of different models, so that multiple models can interact at the feature level. Through multi-model joint contrast learning, each network can learn better feature expression, thereby improving the effect of image classification and downstream semantic recognition tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of image classification in information technology, and in particular to an image classification method and system based on multi-model joint contrast learning. Background Art

[0002] Image classification is a fundamental visual task whose goal is to assign a category label to a single image. This process can be implemented by a convolutional neural network model, which outputs a category probability distribution. During the training phase, the joint learning of multiple models can improve the generalization of each model, thereby improving image classification performance. The earliest deep mutual learning guided one model to learn the output category probability distribution of another model. This point-to-point teaching model can achieve better results than training each model individually. Online ensemble distillation methods integrate the category probability distributions of multiple models to form a virtual teacher role. This ensemble distribution is responsible for imparting knowledge to each model. Subsequent methods have mainly considered how to better integrate knowledge from multiple models based on this framework, such as using self-attention mechanisms and Lagrange multipliers to optimize weighting coefficients to obtain more informative virtual teachers.

[0003] While existing multi-model joint learning methods can effectively improve the performance of individual models within a group, they only consider the transfer of knowledge at the level of category output probabilities. The difference lies in the way different probability distributions are integrated, and they ignore the more informative feature-level knowledge. Multi-model joint learning methods based on category probabilities only consider the final output of the training set as the only knowledge, resulting in a failure to learn the essential classification basis and feature representations, leading to severe model overfitting and unsatisfactory results for downstream semantic recognition tasks. Summary of the Invention

[0004] This paper aims to address the problem of model overfitting in existing multi-model joint learning techniques based on class probability. It proposes an image classification method based on multi-model joint contrastive learning. This method uses contrastive learning between multiple models to achieve joint learning at the feature level and transfers the similarity probability distribution, thereby improving the feature learning quality and image recognition performance of each model in the group.

[0005] Specifically, the present invention provides an image classification method based on multi-model joint contrast learning, which includes:

[0006] Step 1: Obtain a group joint learning model composed of multiple neural networks, and obtain a training set composed of multiple training images that have been labeled with image categories;

[0007] Step 2: a training image in the training set is used as an anchor sample, a training image in the training set with the same category as the anchor sample is used as a positive sample, and the remaining training images are used as negative samples; the anchor sample, the positive sample, and the negative sample are respectively input into the neural network to obtain an anchor vector, a positive vector, and a negative vector; the similarity between the anchor vector and the positive vector is maximized and the similarity between the anchor vector and the negative vector is minimized by using the cross entropy error to obtain a first error of the neural network, and the errors of all the neural networks in the group are combined to obtain a first contrastive learning error;

[0008] Step 3: Multiply the anchor vector of one neural network in the group by the negative vector of another neural network to obtain a similarity distribution between the two. Use the cross entropy error to maximize the similarity between the anchor vector and the positive vector in the similarity distribution, and minimize the similarity between the anchor vector and the negative vector in the similarity distribution to obtain a second error between the two. Collect the second errors of all pairwise neural networks in the group to obtain a second contrastive learning error.

[0009] Step 4: Based on the similarity distribution, the minimum square error is used to approximate the probability distributions of the two neural networks to obtain the minimum square error function;

[0010] Step 5: Construct a total error based on the first contrastive learning error, the second contrastive learning error, and the least square error function, iteratively train the group according to the total error until the total error converges or reaches a preset number of iterations, save the group as an image classification model, and perform the image classification task.

[0011] The image classification method based on multi-model joint contrast learning, wherein

[0012] The step 1 includes: obtaining M (M≥2) neural networks Conduct joint learning as this group;

[0013] Step 2 includes: for the input training image x, M neural networks output M image features d is the dimension of the feature vector; given an input image x 0 As an anchor sample, it will be compared with x 0 Images of the same category x 1 is a positive sample, K images is a negative sample;

[0014] Input the training images into M neural networks In the neural network, the corresponding anchor vector F is obtained m 0 , positive vector and negative vectors Where m represents the mth network in the group network; the similarity distribution P between the anchor vector and the comparison vector is calculated using the point multiplication method of the following formula 1 m , the comparison vector includes positive vectors and negative vectors:

[0015]

[0016] The first error of Equation 2 is used to maximize the similarity between the anchor vector and the positive vector and minimize the similarity between the anchor vector and the negative vector:

[0017]

[0018] For the M neural networks in the group, each network performs the same pattern of contrastive learning, resulting in the following first contrastive learning error:

[0019]

[0020] This step 3 includes:

[0021] Select 2 networks f from M networks a and f b , the anchor vector is generated from f a , the contrast vector is generated from f b , use formula 4 to calculate the similarity distribution:

[0022]

[0023] Use the second error to maximize the similarity between the anchor vector and the positive vector, and minimize the similarity between the anchor vector and the negative vector:

[0024]

[0025] For the M neural networks in the group, every two networks perform interactive contrastive learning in the same mode, and the following second contrastive learning error is obtained:

[0026]

[0027] The above contrastive learning error is calculated for M networks, and the contrastive learning error between networks is calculated. For any two networks a and b, where a is not equal to b, interactive contrastive learning is performed using Formula 5.

[0028] This step 4 includes:

[0029] After applying interactive contrastive learning to the two combinations within M networks, a contrastive similarity distribution is generated. The probability distributions of the two networks are approximated to each other using the least square error function:

[0030]

[0031] This step 5 includes:

[0032] The total error is minimized using a stochastic gradient descent optimizer. After training, the discriminative feature space of the group joint learning model is obtained, in which each image category has a category center. Given an input sample, by comparing the distance between the input sample and the category center, the category closest to the category center is selected as the image category of the input sample.

[0033] The image classification method based on multi-model joint contrast learning, wherein the neural network can adopt convolutional neural networks VGGNet, ResNet and DenseNet.

[0034] The image classification method based on multi-model joint contrast learning, wherein step 5 includes:

[0035] In the process of minimizing the total error using the stochastic gradient descent optimizer, a training image in the training set with a different category from the previous iteration is selected as the anchor sample at each iteration.

[0036] The present invention also proposes an image classification system based on multi-model joint contrast learning, which includes:

[0037] An initialization module is used to obtain a group joint learning model composed of multiple neural networks and a training set composed of multiple training images that have been labeled with image categories;

[0038] A first contrastive learning module is configured to use a training image in the training set as an anchor sample, training images in the training set that are of the same category as the anchor sample as positive samples, and the remaining training images as negative samples; input the anchor sample, the positive sample, and the negative sample into the neural network respectively to obtain an anchor vector, a positive vector, and a negative vector; maximize the similarity between the anchor vector and the positive vector and minimize the similarity between the anchor vector and the negative vector by using a cross-entropy error to obtain a first error of the neural network; and aggregate the errors of all the neural networks in the group to obtain a first contrastive learning error;

[0039] A second contrastive learning module is configured to multiply the anchor vector of one neural network in the group by the negative vector of another neural network to obtain a similarity distribution between the two, use a cross entropy error to maximize the similarity between the anchor vector and the positive vector in the similarity distribution, and minimize the similarity between the anchor vector and the negative vector in the similarity distribution to obtain a second error between the two, and aggregate the second errors of all pairwise neural networks in the group to obtain a second contrastive learning error;

[0040] An error calculation module is used to use the least square error to approximate the probability distributions of the two neural networks based on the similarity distribution to obtain a least square error function;

[0041] The training iteration module is used to construct a total error based on the first contrast learning error, the second contrast learning error and the least square error function, iteratively train the group according to the total error until the total error converges or reaches a preset number of iterations, save the group as an image classification model, and perform an image classification task.

[0042] The image classification method based on multi-model joint contrast learning, wherein

[0043] The initial module includes: obtaining M (M≥2) neural networks Conduct joint learning as this group;

[0044] The first contrastive learning module includes: for the input training image x, M neural networks output M image features d is the dimension of the feature vector; given an input image x 0 As an anchor sample, it will be compared with x 0 Images of the same category x 1 is a positive sample, K images is a negative sample;

[0045] Input the training images into M neural networks In the neural network, the corresponding anchor vector F is obtained m 0 , positive vector and negative vectors Where m represents the mth network in the group network; the similarity distribution P between the anchor vector and the comparison vector is calculated using the point multiplication method of the following formula 1 m , the comparison vector includes positive vectors and negative vectors:

[0046]

[0047] The first error of Equation 2 is used to maximize the similarity between the anchor vector and the positive vector and minimize the similarity between the anchor vector and the negative vector:

[0048]

[0049] For the M neural networks in the group, each network performs the same pattern of contrastive learning, resulting in the following first contrastive learning error:

[0050]

[0051] The second contrastive learning module includes:

[0052] Select 2 networks f from M networks a and f b , the anchor vector is generated from f a , the contrast vector is generated from f b , use formula 4 to calculate the similarity distribution:

[0053]

[0054] Use the second error to maximize the similarity between the anchor vector and the positive vector, and minimize the similarity between the anchor vector and the negative vector:

[0055]

[0056] For the M neural networks in the group, every two networks perform interactive contrastive learning in the same mode, and the following second contrastive learning error is obtained:

[0057]

[0058] The above contrastive learning error is calculated for M networks, and the contrastive learning error between networks is calculated. For any two networks a and b, where a is not equal to b, interactive contrastive learning is performed using Formula 5.

[0059] The error calculation module includes:

[0060] After applying interactive contrastive learning to the two combinations within M networks, a contrastive similarity distribution is generated. The probability distributions of the two networks are approximated to each other using the least square error function:

[0061]

[0062] The training iteration module includes:

[0063] The total error is minimized using a stochastic gradient descent optimizer. After training, the discriminative feature space of the group joint learning model is obtained, in which each image category has a category center. Given an input sample, by comparing the distance between the input sample and the category center, the category closest to the category center is selected as the image category of the input sample.

[0064] The image classification method based on multi-model joint contrast learning, wherein the neural network can adopt convolutional neural networks VGGNet, ResNet and DenseNet.

[0065] The image classification method based on multi-model joint contrast learning, wherein the training iteration module includes:

[0066] In the process of minimizing the total error using the stochastic gradient descent optimizer, a training image in the training set with a different category from the previous iteration is selected as the anchor sample at each iteration.

[0067] The present invention also proposes a storage medium for storing a program for executing any one of the image classification methods based on multi-model joint contrast learning.

[0068] The present invention also proposes a client for any image classification system based on multi-model joint contrast learning.

[0069] From the above scheme, it can be seen that the advantages of the present invention are:

[0070] Compared with other existing multi-model joint learning methods, the present invention can enable each model in the group to achieve higher performance without changing the time overhead of inference. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 Flowchart of the present invention. DETAILED DESCRIPTION

[0072] The defects of the prior art that are prone to overfitting and poor performance in semantic recognition tasks are caused by the knowledge form of previous algorithm modeling, which only considers the highly abstract information distribution of output category probability. Therefore, it cannot essentially reflect the model's reasoning process and basis for the image, which makes the model prone to overfitting. The present invention solves this technical defect by using a method of contrastive learning with multiple models. The present invention uses contrastive learning to make samples of the same category close in the feature space and samples of different categories far away in the feature space, thereby obtaining a discriminative representation space. In the scenario of multi-model joint learning, contrastive learning can establish a relationship between the feature vectors of different models, thereby interacting with multiple models at the feature level. Through multi-model joint contrastive learning, each network can learn better feature expression, thereby improving the effect of image classification and downstream semantic recognition tasks, such as image target detection and image semantic pixel segmentation. In order to achieve the above technical effects, the present invention proposes the following key technical points:

[0073] Key Point 1: This paper proposes a multi-model joint contrastive learning algorithm to guide different models to interact at the feature level. This technically enables the model to learn a more discriminative embedding space, improving feature learning capabilities.

[0074] Key point 2: The present invention mutually transfers the comparative similarity probability distributions between multiple models; in terms of technical effect, it can enable each model in the group to learn the structured feature relationships of other models, improve the robustness of the feature space structure, and further improve the quality of multi-model joint learning at the feature level.

[0075] In order to make the above features and effects of the present invention more clearly understood, the following embodiments are given and described in detail with reference to the accompanying drawings. Figure 1 As shown, the present invention includes the following steps:

[0076] (1) Neural network inference generates feature vectors of images

[0077] Given M (M≥2) neural networks As a group, joint learning is performed. For the input image x, M neural networks output M image features. d is the dimension of the feature vector. The neural network can adopt convolutional neural network VGGNet, ResNet and DenseNet.

[0078] (2) Use contrastive learning to obtain the discriminative space for each model

[0079] For image classification, the present invention defines images from the same category as positive sample pairs and images from different categories as negative sample pairs. Given an input image x 0 As anchor samples, and considering that the training process gradually inputs different data streams, the anchor samples will be randomly replaced with another different type in each iteration. 0 Images of the same category x 1 is a positive sample, K images is a negative sample. It should be noted that the class label information of the present invention is reflected in the selection of positive and negative samples. Samples of the same class are positive samples, and samples of different classes are negative samples. Through the following contrastive learning, samples of the same class are brought closer together, and samples of different classes are separated, thus obtaining a discriminative feature space.

[0080] Input these images into M neural networks In the neural network, the corresponding anchor vector F is obtained m 0 , positive vector and negative vectors Here m represents the mth network in the group network, and the positive vector and the negative vector are collectively referred to as the comparison vector. The similarity distribution P between the anchor vector and the comparison vector is calculated using the dot multiplication method of the following formula 1: m , the comparison vector includes positive vectors and negative vectors. The positive vector can be regarded as the vector that matches the anchor vector, and the negative vector can be regarded as the vector that does not match the anchor vector:

[0081]

[0082] The cross entropy error of Equation 2 is used to maximize the similarity between the anchor vector and the positive vector and minimize the similarity between the anchor vector and the negative vector:

[0083]

[0084] Minimizing the error in Formula 2 maximizes the similarity between the anchor vector and the positive vector, and minimizes the similarity between the anchor vector and the negative vector. Similarity is uniformly defined as the dot product of the two vectors.

[0085] For the M neural networks in the group, each network performs the same pattern of contrastive learning, resulting in the following contrastive learning error:

[0086]

[0087] The above contrastive learning error is the sum of M contrastive learning errors. The M errors optimize M neural networks respectively, where each error corresponds to formula (2). This error can enable the neural network to learn a more discriminative feature space, that is, the anchor sample is closer to the positive sample and farther away from the negative sample.

[0088] (3) Interactive comparative learning of multiple models

[0089] Traditional contrastive learning cannot model the relationship between networks for joint learning, because the contrastive distribution is learned from the discriminative feature space of the network. In order to promote the interaction at the feature level within M networks, interactive contrastive learning is introduced. Select 2 networks f from M networks. a and f b , the anchor vector is generated from f a , the contrast vector is generated from f b , use the same rules as formula (1) to calculate the similarity distribution:

[0090]

[0091] Use the cross entropy error to maximize the similarity between the anchor vector and the positive vector, and minimize the similarity between the anchor vector and the negative vector:

[0092]

[0093] For the M neural networks in the group, every two networks perform interactive contrastive learning in the same mode, resulting in the following contrastive learning error:

[0094]

[0095] The above contrastive learning error is calculated for M networks, with contrastive learning errors between each network. For any two networks a and b, where a is not equal to b, interactive contrastive learning is performed using formula (5), so that the network learns a discriminative feature space.

[0096] (IV) Comparison of Probability Distribution Distillation between Multiple Models

[0097] After applying interactive contrastive learning to the two combinations within the M networks, a contrast probability (similarity) distribution is generated. The present invention further compares the probability distributions of the two networks and approximates each other using the least square error function:

[0098]

[0099] The above-mentioned minimum square error is used to approximate the two similarity values ​​so that the two are close, thereby allowing the two networks to learn the similarity values ​​from each other.

[0100] (V) Optimization error term

[0101] The three error functions (3)(6)(7) mentioned above are optimized sequentially to obtain the total error. The stochastic gradient descent optimizer is used to minimize the error function, making the model show excellent image classification performance. After training, the network will obtain a discriminative feature space, and each category will have a category center. Given an input sample, the distance to each category center is calculated. The sample belongs to the category with the closest distance to the category center.

[0102] The following is a system embodiment corresponding to the above method embodiment. This embodiment can be implemented in conjunction with the above embodiment. The relevant technical details mentioned in the above embodiment are still valid in this embodiment and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiment.

[0103] The present invention also proposes an image classification system based on multi-model joint contrast learning, which includes:

[0104] An initialization module is used to obtain a group joint learning model composed of multiple neural networks and a training set composed of multiple training images that have been labeled with image categories;

[0105] A first contrastive learning module is configured to use a training image in the training set as an anchor sample, training images in the training set that are of the same category as the anchor sample as positive samples, and the remaining training images as negative samples; input the anchor sample, the positive sample, and the negative sample into the neural network respectively to obtain an anchor vector, a positive vector, and a negative vector; maximize the similarity between the anchor vector and the positive vector and minimize the similarity between the anchor vector and the negative vector by using a cross-entropy error to obtain a first error of the neural network; and aggregate the errors of all the neural networks in the group to obtain a first contrastive learning error;

[0106] A second contrastive learning module is configured to multiply the anchor vector of one neural network in the group by the negative vector of another neural network to obtain a similarity distribution between the two, use a cross entropy error to maximize the similarity between the anchor vector and the positive vector in the similarity distribution, and minimize the similarity between the anchor vector and the negative vector in the similarity distribution to obtain a second error between the two, and aggregate the second errors of all pairwise neural networks in the group to obtain a second contrastive learning error;

[0107] An error calculation module is used to use the least square error to approximate the probability distributions of the two neural networks based on the similarity distribution to obtain a least square error function;

[0108] The training iteration module is used to construct a total error based on the first contrast learning error, the second contrast learning error and the least square error function, iteratively train the group according to the total error until the total error converges or reaches a preset number of iterations, save the group as an image classification model, and perform an image classification task.

[0109] The image classification method based on multi-model joint contrast learning, wherein

[0110] The initial module includes: obtaining M (M≥2) neural networks Conduct joint learning as this group;

[0111] The first contrastive learning module includes: for the input training image x, M neural networks output M image features d is the dimension of the feature vector; given an input image x 0 As an anchor sample, it will be compared with x 0 Images of the same category x 1 is a positive sample, K images is a negative sample;

[0112] Input the training images into M neural networks In the neural network, the corresponding anchor vector F is obtained m 0 , positive vector and negative vectors Where m represents the mth network in the group network; the similarity distribution P between the anchor vector and the comparison vector is calculated using the point multiplication method of the following formula 1 m , the comparison vector includes positive vectors and negative vectors:

[0113]

[0114] The first error of Equation 2 is used to maximize the similarity between the anchor vector and the positive vector and minimize the similarity between the anchor vector and the negative vector:

[0115]

[0116] For the M neural networks in the group, each network performs the same pattern of contrastive learning, resulting in the following first contrastive learning error:

[0117]

[0118] The second contrastive learning module includes:

[0119] Select 2 networks f from M networks a and f b , the anchor vector is generated from f a , the contrast vector is generated from f b , use formula 4 to calculate the similarity distribution:

[0120]

[0121] Use the second error to maximize the similarity between the anchor vector and the positive vector, and minimize the similarity between the anchor vector and the negative vector:

[0122]

[0123] For the M neural networks in the group, every two networks perform interactive contrastive learning in the same mode, and the following second contrastive learning error is obtained:

[0124]

[0125] The above contrastive learning error is calculated for M networks, and the contrastive learning error between networks is calculated. For any two networks a and b, where a is not equal to b, interactive contrastive learning is performed using Formula 5.

[0126] The error calculation module includes:

[0127] After applying interactive contrastive learning to the two combinations within M networks, a contrastive similarity distribution is generated. The probability distributions of the two networks are approximated to each other using the least square error function:

[0128]

[0129] The training iteration module includes:

[0130] The total error is minimized using a stochastic gradient descent optimizer. After training, the discriminative feature space of the group joint learning model is obtained, in which each image category has a category center. Given an input sample, by comparing the distance between the input sample and the category center, the category closest to the category center is selected as the image category of the input sample.

[0131] The image classification method based on multi-model joint contrast learning, wherein the neural network can adopt convolutional neural networks VGGNet, ResNet and DenseNet.

[0132] The image classification method based on multi-model joint contrast learning, wherein the training iteration module includes:

[0133] In the process of minimizing the total error using the stochastic gradient descent optimizer, a training image in the training set with a different category from the previous iteration is selected as the anchor sample at each iteration.

[0134] The present invention also proposes a storage medium for storing a program for executing any one of the image classification methods based on multi-model joint contrast learning.

[0135] The present invention also proposes a client for any image classification system based on multi-model joint contrast learning.

Claims

1. An image classification method based on multi-model joint contrast learning, characterized in that: include: Step 1: Get M neural networks Perform joint learning as a group joint learning model, where M is an integer greater than or equal to 2, and obtain a training set consisting of multiple training images that have been labeled with image categories; Step 2: For the input training image x, M neural networks output M image features d is the dimension of the feature vector; given an input image x 0 As an anchor sample, it will be compared with x 0 Images of the same category x 1 is a positive sample, K images is a negative sample; Input the training images into M neural networks In the neural network, the corresponding anchor vector is obtained Positive vector and negative vectors Where m represents the mth network in the group network; the similarity distribution p between the anchor vector and the comparison vector is calculated using the point multiplication method of the following formula 1 m , the comparison vector includes positive vectors and negative vectors: The first error of Equation 2 is used to maximize the similarity between the anchor vector and the positive vector and minimize the similarity between the anchor vector and the negative vector: For the M neural networks in the group, each network performs the same pattern of contrastive learning, resulting in the following first contrastive learning error: Step 3: Select 2 networks f from M networks a and f b , the anchor vector is generated from f a , the contrast vector is generated from f b , use formula 4 to calculate the similarity distribution: Use the second error to maximize the similarity between the anchor vector and the positive vector, and minimize the similarity between the anchor vector and the negative vector: For the M neural networks in the group, every two networks perform interactive contrastive learning in the same mode, and the following second contrastive learning error is obtained: The above contrastive learning error is calculated for M networks, and the contrastive learning error between networks is calculated. For any two networks a and b, where a is not equal to b, interactive contrastive learning is performed using Formula 5. Step 4: After applying interactive contrastive learning to the two combinations within the M networks, a contrastive similarity distribution is generated. The probability distributions of the two networks are approximated to each other using the least square error function: Step 5: Construct a total error based on the first contrastive learning error, the second contrastive learning error, and the least square error function, and minimize the total error using a stochastic gradient descent optimizer. After the training is completed, a discriminative feature space of the group joint learning model is obtained, in which each image category has a category center. Given an input sample, by comparing the distance between the input sample and the center of each category, the category closest to the category center is selected as the image category of the input sample.

2. The image classification method based on multi-model joint contrast learning according to claim 1, characterized in that: The neural network can adopt convolutional neural network VGGNet, ResNet and DenseNet.

3. The image classification method based on multi-model joint contrast learning according to claim 1, characterized in that: This step 5 includes: In the process of minimizing the total error using the stochastic gradient descent optimizer, a training image in the training set with a different category from the previous iteration is selected as the anchor sample at each iteration.

4. An image classification system based on multi-model joint contrastive learning, characterized in that: include: Initial module, obtain M neural networks Perform joint learning as a group joint learning model, where M is an integer greater than or equal to 2, and obtain a training set consisting of multiple training images that have been labeled with image categories; In the first contrastive learning module, for the input training image x, M neural networks output M image features d is the dimension of the feature vector; given an input image x 0 As an anchor sample, it will be compared with x 0 Images of the same category x 1 is a positive sample, K images is a negative sample; Input the training images into M neural networks In the neural network, the corresponding anchor vector is obtained Positive vector and negative vectors Where m represents the mth network in the group network; the similarity distribution p between the anchor vector and the comparison vector is calculated using the point multiplication method of the following formula 1 m , the comparison vector includes positive vectors and negative vectors: The first error of Equation 2 is used to maximize the similarity between the anchor vector and the positive vector and minimize the similarity between the anchor vector and the negative vector: For the M neural networks in the group, each network performs the same pattern of contrastive learning, resulting in the following first contrastive learning error: The second contrastive learning module selects 2 networks f from M networks a and f b , the anchor vector is generated from f a , the contrast vector is generated from f b , use formula 4 to calculate the similarity distribution: Use the second error to maximize the similarity between the anchor vector and the positive vector, and minimize the similarity between the anchor vector and the negative vector: For the M neural networks in the group, every two networks perform interactive contrastive learning in the same mode, and the following second contrastive learning error is obtained: The above contrastive learning error is calculated for M networks, and the contrastive learning error between networks is calculated. For any two networks a and b, where a is not equal to b, interactive contrastive learning is performed using Formula 5. The error calculation module generates a comparative similarity distribution after applying interactive comparative learning to the two combinations within M networks The probability distributions of the two networks are approximated to each other using the least square error function: The training iteration module constructs a total error based on the first contrastive learning error, the second contrastive learning error and the least square error function, and minimizes the total error using a stochastic gradient descent optimizer. After the training is completed, the discriminative feature space of the group joint learning model is obtained, in which each image category has a category center. Given an input sample, by comparing the distance between the input sample and the center of each category, the category with the closest distance to the category center is selected as the image category of the input sample.

5. The image classification system based on multi-model joint contrastive learning according to claim 4, characterized in that: The neural network can adopt convolutional neural network VGGNet, ResNet and DenseNet.

6. The image classification system based on multi-model joint contrastive learning according to claim 4, characterized in that: The training iteration module includes: In the process of minimizing the total error using the stochastic gradient descent optimizer, a training image in the training set with a different category from the previous iteration is selected as the anchor sample at each iteration.

7. A storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image classification method based on multi-model joint contrast learning as described in any one of claims 1 to 3.

8. A client, used in the image classification system based on multi-model joint contrast learning as described in any one of claims 4 to 6.

Images