Image classification model construction and training method based on difficulty sample correlation learning

Abstract

The application relates to the technical field of computer vision, and particularly discloses an image classification model construction and training method based on difficult-easy sample correlation learning, which comprises the following steps: processing sample images in a training set and constructing an image classification model; training the image classification model by using all the processed sample images in the training set to obtain a pre-training image classification model; performing inference on all the processed sample images in the training set by using the pre-training image classification model, and dividing the training set into a difficult sample set and an easy sample set; and optimizing and training the pre-training image classification model based on a correlation learning mechanism of the difficult sample set and the easy sample set, updating the difficult sample set and the easy sample set after each round of optimization and training, and obtaining a final image classification model after multiple rounds of optimization and training. The application can reduce the bias caused by class distribution imbalance, enhance the capture learning of difficult sample features by the model, and improve the precision of image classification and recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and more particularly, to an image classification model construction and training method based on difficult-easy sample correlation learning. BACKGROUND

[0002] In recent years, the rapid development of science and technology has a positive driving effect on the sharp increase in data volume and data availability in various fields. The improvement of GPU hardware performance and the continuous improvement of network structure enable deep neural networks to be effectively trained and widely used in the field of computer vision. In deep learning, data is an indispensable element.

[0003] In the image classification task of deep learning, we face a series of challenges. First, unlike the situation where the samples of each class are uniformly distributed in the laboratory data set, the class imbalance problem exists in the real world large-scale data set, resulting in rich samples of some classes and scarce samples of other classes. Secondly, the training samples in the real scene not only show high inter-class similarity in the feature space of the classification model, but also have obvious intra-class differences. The features of some samples are easy to learn, while the feature expression of other difficult samples is often unsatisfactory. In order to solve the above problems, two kinds of solutions are currently adopted: one is to use different resampling methods at the data training level, and the other is to improve the model or loss function at the classifier level. However, in these methods, the examination and measurement of difficult and easy samples in the data set are usually independent and irrelevant, which cannot effectively balance the learning process of difficult and easy samples. Therefore, the present application provides an image classification model construction and training method based on difficult-easy sample correlation learning. SUMMARY

[0004] The purpose of the present application is to overcome the deficiencies in the prior art and provide an image classification model construction and training method based on difficult-easy sample correlation learning, which can improve the robustness of the model in image recognition.

[0005] As a first aspect of the present application, an image classification model construction and training method based on difficult-easy sample correlation learning is provided, comprising the following steps:

[0006] Step S1: obtaining sample images in the training set and constructing an image classification model;

[0007] Step S2: preprocessing the sample images in the training set to obtain processed sample images;

[0008] Step S3: using all the processed sample images in the training set to preliminarily train the constructed image classification model to obtain a pre-trained image classification model;

[0009] Step S4: using the pre-trained image classification model to infer all processed sample images in the training set, and then dividing the training set into a difficult sample set and an easy sample set according to the sample image inference results;

[0010] Step S5: based on the associated learning mechanism of the difficult sample set and the easy sample set, the pre-trained image classification model is trained, after each round of training, the image classification model after the current training is used to infer the difficult sample images in the current difficult sample set and the easy sample images in the current easy sample set, and the current difficult sample set and the current easy sample set are updated according to the inference results of the difficult sample images and the inference results of the easy sample images, and after multiple rounds of training, the final image classification model is obtained.

[0011] Further, the constructing image classification model further comprises:

[0012] The framework of the image classification model comprises a feature extraction module and a feature classification module;

[0013] The feature extraction module is responsible for performing multi-layer convolution on the input sample image and flattening to obtain the feature vector of the input sample image;

[0014] The feature classification module is responsible for mapping the feature vector of the input sample image to a k-dimensional feature vector through full connection, and passing through a Softmax activation function to obtain the confidence of the input sample image being predicted as each class of k classes, wherein k is the total number of classes of all sample images in the training set.

[0015] Further, the using all processed sample images in the training set to preliminarily train the constructed image classification model to obtain a pre-trained image classification model further comprises:

[0016] In the pre-training process of the image classification model, according to the distance between the class predicted by the input sample image through the image classification model and the real class, an Euler sorting loss function ERLoss is designed as the loss function of the image classification model;

[0017] Based on the position of the real class in the predicted class among all classes, the confidence of the input sample image being predicted as the real class is extended to a complex expression according to Euler's formula:

[0018]

[0019] Wherein, p μc is the confidence of the sample image μ being predicted as the real class c by the image classification model, σ μc is the descending rank of the real class c in the predicted class of the sample image μ among all classes, and σ μc= {0, 1,..., k-1}, the square of imaginary number i is -1, and k is the total number of classes of all sample images in the training set;

[0020] In combination with the cross-entropy and the modulus operation rule of complex numbers, the Euler ranking loss function ERLoss of the image classification model is described as:

[0021]

[0022] wherein M is the total number of sample images in the training set; γ μc is a symbol function, taking 0 or 1, if the real class of the sample image μ is equal to c, γ μc takes 1, otherwise, γ μc takes 0.

[0023] Further, the using the pre-trained image classification model to perform inference on all processed sample images in the training set, and then dividing the training set into a difficult sample set and an easy sample set according to the sample image inference results, further comprises:

[0024] After obtaining the pre-trained image classification model, using the pre-trained image classification model to perform inference on all processed sample images in the training set;

[0025] For a sample image in the training set, if the confidence of the real class predicted by the pre-trained image classification model is not less than 0.5, the sample image is determined as an easy sample image, otherwise, it is determined as a difficult sample image, thereby dividing the training set into a difficult sample set and an easy sample set.

[0026] Further, the optimization training of the pre-trained image classification model based on the correlation learning mechanism of the difficult sample set and the easy sample set comprises:

[0027] In each round of optimization training, the difficult sample images in the current difficult sample set and the easy sample images in the current easy sample set are mixed in equal proportions and sent in multiple batches into the current image classification model for optimization training; wherein in the optimization training process of the current image classification model, a batch of m easy sample images correspondingly obtain easy sample features l i , wherein i = 1, 2,..., m; at the same time, a batch of n difficult sample images correspondingly obtain difficult sample features u j , wherein j = 1, 2,..., n;

[0028] The similarity s i between the easy sample features l j and the difficult sample features u ijand get the similarity matrix between easy sample features and difficult sample features

[0029]

[0030] where ψ(·, ·) is a similarity measure function, and the Euclidean distance, cosine similarity or radial basis function is selected;

[0031] Based on the similarity matrix S, the easy sample features l i and the difficult sample features u j are associated to obtain the association probability matrix and the difficult sample features u j and the easy sample features l i are associated to obtain the association probability matrix where, for the easy sample features l i , the association probability p j of the current batch to any difficult sample feature u ij is calculated by using the Softmax function to normalize the jth column of the similarity matrix S; for the difficult sample features u j , the association probability q i of the current batch to any easy sample feature l ji is calculated by using the Softmax function to normalize the ith column of the similarity matrix S; the calculation formula is as follows:

[0032]

[0033] Based on the association probability matrix P between the easy sample features and the difficult sample features and the association probability matrix Q between the difficult sample features and the easy sample features, the association relationship QP between the difficult sample features in the feature space formed by all sample features in the current batch is calculated;

[0034] According to the association relationship QP between the difficult sample features in the feature space, the association probability matrix Q between the difficult sample features and the easy sample features, and the classification vector matrix Y of all easy sample features , the classification vector y j of any difficult sample feature u in the current batch is calculated, and the classification vector matrix Y of all difficult sample features in the current batch is obtained The calculation formula is as follows:

[0035]

[0036] where r jj′ is the difficult sample feature uj′ For difficult sample features u j The relationship between them is given by A, which is a matrix composed of classification vectors obtained after all easy sample features have passed through the current feature classification module.

[0037] Furthermore, it also includes:

[0038] The loss of the current image classification model consists of two parts: the loss for difficult sample images and the loss for easy sample images. Both the loss for difficult and easy sample images are Euler ranking losses of the classification vectors and true labels of the sample images. Specifically, the feature l of any easy sample in the current batch... i The classification vector is From any easy sample feature l i It is obtained directly through the feature classification module of the current image classification model;

[0039]

[0040] Among them, L easy and L hard σ represents the loss for easy sample images and the loss for hard sample images, respectively; λ is the influence factor of the loss for hard sample images, which defaults to 1; m is the number of easy sample images in the easy sample set, and n is the number of hard sample images in the hard sample set; ic Let σ be the descending ranking of the true class c among all classes in the predicted class of sample image i. jc For the predicted category of sample image j, rank the true category c in descending order among all classes; γ ic γ is a sign function, taking the value 0 or 1. If the true class of sample image i is equal to c, then γ ic Take 1, otherwise, γ ic Set to 0; γ jc γ is a sign function, taking the value 0 or 1. If the true class of sample image j is equal to c, then γ jc Take 1, otherwise, γ jc Take 0.

[0041] Furthermore, after each round of optimization training, the currently optimized image classification model is used to infer the difficult sample images in the current difficult sample set and the easy sample images in the current easy sample set. The current difficult sample set and the current easy sample set are updated based on the inference results of the difficult sample images and the easy sample images. After multiple rounds of optimization training, the final image classification model is obtained, which also includes:

[0042] For a difficult sample image in the current difficult sample set or an easy sample image in the current easy sample set, if the confidence level of the image classification model after current optimization training predicts it as the true class is not less than 0.5, then the sample image is determined to be an easy sample image; otherwise, it is determined to be a difficult sample image. Thus, the current difficult sample set and the current easy sample set are updated, and a new round of optimization training is performed. The optimization is iterated until the image classification model after current optimization training meets the target.

[0043] Furthermore, the easily sampled features l in the computational feature space i Features of difficult samples u j Pairwise similarity s ij It also includes:

[0044] Cosine similarity was chosen as an easy-to-sample feature in the feature space. i Features of difficult samples u j The similarity measure function between pairs of objects is calculated using the following formula:

[0045]

[0046] Among them, similarity s ij A value close to 1 indicates an easy sample feature. i Features of difficult samples u j Similarity; similarity s ij A value close to -1 indicates an easily accessible sample feature. i Features of difficult samples u j on the contrary.

[0047] The image classification model construction and training method based on the association learning of difficult and easy samples provided by this invention has the following advantages: it can reduce the bias caused by class imbalance, enhance the model's ability to capture and learn the features of difficult samples, and improve the accuracy of image classification and recognition. Attached Figure Description

[0048] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the following detailed description to explain the invention, but do not constitute a limitation thereof.

[0049] Figure 1 This invention provides an overall flowchart of an image classification model construction and training method based on the association learning of easy and difficult samples.

[0050] Figure 2 This is a flowchart illustrating a specific implementation method for optimizing and training a pre-trained image classification model, as provided by the present invention. Detailed Implementation

[0051] For further illustrating the technical means and effects taken by the present application to achieve the predetermined inventive objectives, the following will combine the drawings and preferred embodiments to specifically explain the implementation, structure, features and effects of the image classification model construction and training method based on hard and easy sample correlation learning according to the present application, as follows. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the present application.

[0052] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0053] In the present embodiment, an image classification model construction and training method based on hard and easy sample correlation learning is provided, as shown in Figure 1 The image classification model construction and training method based on hard and easy sample correlation learning comprises the following steps:

[0054] Step S1: obtaining sample images in a training set and constructing an image classification model;

[0055] Preferably, the constructing an image classification model further comprises:

[0056] The basic framework of the image classification model comprises a feature extraction module and a feature classification module;

[0057] The feature extraction module is responsible for performing multi-layer convolution on the input sample image and flattening to obtain the feature vector of the input sample image. Specifically, the feature extraction module selects the MobileNetV2 network which discards the last global pooling and linear dimension reduction layer.

[0058] The feature classification module is responsible for mapping the feature vector of the input sample image to a k-dimensional feature vector through full connection and passing through a Softmax activation function to obtain the confidence of the input sample image being predicted as each class of k classes, wherein k is the total number of classes of all sample images in the training set.

[0059] Step S2: preprocessing the sample images in the training set to obtain processed sample images;

[0060] Step S3: using all the processed sample images in the training set to preliminarily train the constructed image classification model to obtain a pre-trained image classification model with a certain level of precision;

[0061] Preferably, the using all the processed sample images in the training set to preliminarily train the constructed image classification model to obtain a pre-trained image classification model further comprises:

[0062] In the image classification model pre-training process, an Euler ranking loss function ERLoss is designed as the loss function of the image classification model according to the distance between the category predicted by the image classification model for the input sample image and the real category thereof;

[0063] Based on the position of the real category in the predicted category among all categories, the confidence of the input sample image being predicted as the real category is extended to a complex expression according to Euler's formula:

[0064]

[0065] Wherein, p μc is the confidence of the sample image μ being predicted as the real category c by the image classification model, σ μc is the descending rank of the real category c in the predicted category of the sample image μ among all categories, σ μc ={0, 1,..., k-1}, the square of the imaginary number i is -1, and k is the total number of categories of all sample images in the training set;

[0066] Combined with cross-entropy and the modulus operation rule of complex numbers, the Euler ranking loss function ERLoss of the image classification model is described as:

[0067]

[0068] Wherein, M is the total number of sample images in the training set; γ μc is a symbol function, taking the value of 0 or 1, if the real category of the sample image μ is equal to c, γ μc takes 1, otherwise, γ μc takes 0.

[0069] Step S4: using the pre-trained image classification model to infer all the processed sample images in the training set, and then dividing the training set into a difficult sample set and an easy sample set according to the sample image inference results;

[0070] Preferably, the step of using the pre-trained image classification model to infer all processed sample images in the training set, and then dividing the training set into a difficult sample set and an easy sample set based on the inference results of the sample images, further includes:

[0071] After obtaining the pre-trained image classification model, the pre-trained image classification model is used to perform inference on all processed sample images in the training set;

[0072] For a sample image in the training set, if the confidence level of the pre-trained image classification model predicting it as the true class is not less than 0.5, then the sample image is determined to be an easy sample image; otherwise, it is determined to be a difficult sample image. Thus, the training set is divided into a difficult sample set and an easy sample set.

[0073] Step S5: The pre-trained image classification model is fine-tuned based on the association learning mechanism of the difficult sample set and the easy sample set. After each round of fine-tuning, the image classification model is used to infer the difficult sample images in the current difficult sample set and the easy sample images in the current easy sample set. The current difficult sample set and the current easy sample set are updated according to the inference results of the difficult sample images and the inference results of the easy sample images. After multiple rounds of fine-tuning, the final image classification model is obtained.

[0074] Preferably, such as Figure 2 As shown, the association learning mechanism based on difficult and easy sample sets is used to optimize and train the pre-trained image classification model, including:

[0075] In each round of optimization training, difficult sample images from the current difficult sample set and easy sample images from the current easy sample set are mixed in equal proportions and fed into the current image classification model in multiple batches for optimization training. During the optimization training of the current image classification model, each batch of m easy sample images is processed by the feature extraction module in the current image classification model to obtain easy sample features l. i Where i = 1, 2, ..., m; simultaneously, a batch of n difficult sample images are processed by the feature extraction module in the current image classification model to obtain the difficult sample features u. j , where j = 1, 2, ..., n;

[0076] Calculate the easy sample features l in the feature space i Features of difficult samples u j Pairwise similarity s ij And obtain the similarity matrix between easy sample features and hard sample features.

[0077]

[0078] where, ψ(·, ·) is a similarity measure function, and the Euclidean distance, cosine similarity or radial basis function is selected; for example, the cosine similarity is selected as the characteristic space of the easy sample feature l i and the difficult sample feature u j The similarity measure function between each other is calculated as follows:

[0079]

[0080] where, the value of the similarity s ij approaches 1, indicating that the easy sample feature l i is similar to the difficult sample feature u j ; the value of the similarity s ij approaches -1, indicating that the easy sample feature l i is opposite to the difficult sample feature u j .

[0081] Based on the similarity matrix S, the association probability matrix P between the easy sample feature l i and the difficult sample feature u j is obtained and the association probability matrix Q between the difficult sample feature u j and the easy sample feature l i is obtained where, for the easy sample feature l i , the association probability p ij to any difficult sample feature u j in the current batch is calculated by using the Softmax function to normalize the jth column of the similarity matrix S; for the difficult sample feature u j , the association probability q i to any easy sample feature l ji in the current batch is calculated by using the Softmax function to normalize the ith column of the similarity matrix S; the calculation formula is as follows:

[0082]

[0083] Based on the association probability matrix P between the easy sample feature and the difficult sample feature and the association probability matrix Q between the difficult sample feature and the easy sample feature, the association relationship QP between the difficult sample features in the characteristic space formed by all sample features in the current batch is calculated;

[0084] According to the association relationship QP between the difficult sample features in the characteristic space, the association probability matrix Q between the difficult sample features and the easy sample features, and the classification vector matrix Y of all easy sample features The three, calculate the current batch of any difficult sample feature u j The classification vector of the difficult sample feature u And get the classification vector matrix of all difficult sample features in the current batch The calculation formula is as follows:

[0085]

[0086] Wherein, r jj′ The correlation of the difficult sample feature u j′ To the difficult sample feature u j , A is a matrix composed of classification vectors obtained by all easy sample features after the current feature classification module;

[0087] The loss of the current image classification model is composed of the loss of the difficult sample image and the loss of the easy sample image, and the loss of the difficult sample image and the loss of the easy sample image are both the Euler order loss of the classification vector of the sample image and the real label; wherein, the classification vector of any easy sample feature l i In the current batch is Directly obtained by any easy sample feature l i Through the feature classification module of the current image classification model;

[0088]

[0089] Wherein, L easy And L hard The loss of the easy sample image and the loss of the difficult sample image; λ is the influence factor of the loss of the difficult sample image, and the default is 1; m is the number of easy sample images in the easy sample set, n is the number of difficult sample images in the difficult sample set; σ ic The real class c in the predicted class of sample image i in descending order in all classes, σ jc The real class c in the predicted class of sample image j in descending order in all classes; γ ic Is a sign function, which takes 0 or 1, if the real class of sample image i is equal to c, γ ic Take 1, otherwise, γ ic Take 0; γ jc Is a sign function, which takes 0 or 1, if the real class of sample image j is equal to c, γ jc Take 1, otherwise, γ jc Take 0.

[0090] Preferably, after each round of the fine-tuning training, the current fine-tuned image classification model is used to infer the difficult sample images in the current difficult sample set and the easy sample images in the current easy sample set, and the current difficult sample set and the current easy sample set are updated according to the inference results of the difficult sample images and the inference results of the easy sample images, and the final image classification model is obtained after the multiple rounds of fine-tuning training, and the method further comprises the following steps of:

[0091] For a difficult sample image in the current difficult sample set or an easy sample image in the current easy sample set, if the confidence of the current fine-tuned image classification model predicting the real category is not less than 0.5, the sample image is determined as an easy sample image, otherwise, the sample image is determined as a difficult sample image, thereby updating the current difficult sample set and the current easy sample set, and a new round of fine-tuning training is performed, and iteration optimization is performed until the current fine-tuned image classification model meets the standard.

[0092] In summary, the image classification model construction and training method based on difficult-easy sample correlation learning provided by the present application can reduce the bias caused by the imbalance of category distribution, enhance the capture and learning of difficult sample features by the model, and improve the accuracy of image classification and recognition.

[0093] The above is only a preferred embodiment of the present application, and does not limit the present application in any form. Although the present application has been disclosed as above with reference to the preferred embodiment, it is not intended to limit the present application. Any person skilled in the art can make some changes or modifications to the above disclosed technical content without departing from the scope of the technical solution of the present application, and any simple modification, equivalent change and modification of the above embodiment according to the technical essence of the present application still belong to the scope of the technical solution of the present application.

Claims

1. A method for constructing and training an image classification model based on the association learning of easy and difficult samples, characterized in that, Includes the following steps: Step S1: Obtain sample images from the training set and construct an image classification model; Step S2: Preprocess the sample images in the training set to obtain processed sample images; Step S3: Use all the processed sample images in the training set to perform preliminary training on the constructed image classification model to obtain a pre-trained image classification model; Step S4: Use the pre-trained image classification model to perform inference on all processed sample images in the training set, and then divide the training set into a difficult sample set and an easy sample set based on the inference results of the sample images; Step S5: The pre-trained image classification model is fine-tuned based on the association learning mechanism of the difficult sample set and the easy sample set. After each round of fine-tuning, the image classification model is used to infer the difficult sample images in the current difficult sample set and the easy sample images in the current easy sample set. The current difficult sample set and the current easy sample set are updated according to the inference results of the difficult sample images and the inference results of the easy sample images. After multiple rounds of fine-tuning, the final image classification model is obtained. The optimization training of the pre-trained image classification model based on the association learning mechanism of difficult and easy sample sets includes: In each round of optimization training, difficult sample images from the current difficult sample set and easy sample images from the current easy sample set are mixed in equal proportions and fed into the current image classification model in multiple batches for optimization training. During the optimization training of the current image classification model, each batch of m easy sample images is processed by the feature extraction module in the current image classification model to obtain easy sample features. ,in Simultaneously, a batch of n difficult sample images are processed by the feature extraction module in the current image classification model to obtain the features of the difficult samples. ,in ; Calculate the features of easy samples in the feature space Features of difficult samples Pairwise similarity And obtain the similarity matrix between easy sample features and hard sample features. ; ， in, It is a similarity measurement function, and you can choose Euclidean distance, cosine similarity, or radial basis function; Based on similarity matrix Obtaining easy sample features Features of difficult samples The correlation probability matrix between and difficult sample features To easily sample features The correlation probability matrix between Among them, for easy sample features In other words, it has the characteristics of any difficult sample in the current batch. Association probability By analyzing the similarity matrix The The columns are calculated using the Softmax function for normalization; features for difficult samples. In other words, it can be used to identify any easy sample feature in the current batch. Association probability By analyzing the similarity matrix The The column is calculated using the Softmax function for normalization; the calculation formula is as follows: ， Based on the correlation probability matrix between easy sample features and difficult sample features The correlation probability matrix between features of difficult samples and features of easy samples Calculate the correlation between pairwise hard sample features in the feature space formed by the features of all samples in the current batch. ; Based on the correlation between the features of pairwise difficult samples in the feature space The correlation probability matrix between features of difficult samples and features of easy samples And the classification vector matrix of all easy sample features. The three methods are used to calculate the features of any difficult sample in the current batch. Classification vectors And obtain the classification vector matrix of features of all difficult samples in the current batch. The calculation formula is as follows: ， in, For features of difficult samples in the current batch feature space Features of difficult samples The relationship between them It is a matrix consisting of the classification vectors obtained after all easy sample features have passed through the current feature classification module.

2. The method for constructing and training an image classification model based on the association learning of easy and difficult samples according to claim 1, characterized in that, The construction of the image classification model also includes: The framework of the image classification model includes a feature extraction module and a feature classification module; The feature extraction module is responsible for performing multi-layer convolution on the input sample image and then flattening it to obtain the feature vector of the input sample image. The feature classification module is responsible for mapping the feature vector of the input sample image into a k-dimensional feature vector through a fully connected layer, and then passing it through the Softmax activation function to obtain the confidence score of the input sample image as predicted into each of the k classes, where k is the total number of classes of all sample images in the training set.

3. The method for constructing and training an image classification model based on the association learning of easy and difficult samples according to claim 2, characterized in that, The step of using all processed sample images in the training set to perform preliminary training on the constructed image classification model to obtain a pre-trained image classification model further includes: During the pre-training process of the image classification model, based on the distance between the category predicted by the image classification model and the true category of the input sample image, the Euler ranking loss function ERLoss is designed as the loss function of the image classification model. Based on the position of the true class among all classes in the predicted category, the confidence score of predicting the input sample image as the true class is expressed in complex form by generalizing Euler's formula: ， in, For sample images Predicted as the true category by the image classification model confidence level For sample images The predicted category is the true category Descending order ranking among all classes , imaginary number The square of is -1, and k is the total number of categories for all sample images in the training set; Combining cross-entropy and the modular arithmetic of complex numbers, the Euler ranking loss function ERLoss for image classification models is described as follows: ， in, The total number of sample images in the training set; This is a sign function, taking the value 0 or 1, if the sample image The true category equals , Take 1, otherwise Take 0.

4. The method for constructing and training an image classification model based on the association learning of easy and difficult samples according to claim 3, characterized in that, The step of using the pre-trained image classification model to infer all processed sample images in the training set, and then dividing the training set into a hard sample set and an easy sample set based on the inference results of the sample images, further includes: After obtaining the pre-trained image classification model, the pre-trained image classification model is used to perform inference on all processed sample images in the training set; For a sample image in the training set, if the confidence level of the pre-trained image classification model predicting it as the true class is not less than 0.5, then the sample image is determined to be an easy sample image; otherwise, it is determined to be a difficult sample image. Thus, the training set is divided into a difficult sample set and an easy sample set.

5. The method for constructing and training an image classification model based on the association learning of easy and difficult samples according to claim 1, characterized in that, Also includes: The loss of the current image classification model consists of two parts: the loss for difficult sample images and the loss for easy sample images. Both the loss for difficult and easy sample images are Euler ranking losses of the classification vectors and ground truth labels of the sample images. Specifically, the loss for any easy sample in the current batch... The classification vector is , by any easy sample feature It is obtained directly through the feature classification module of the current image classification model; ， in, and These represent the losses for easy sample images and the losses for difficult sample images, respectively. This is the influence factor for the loss of hard sample images, with a default value of 1. The number of easy sample images in the easy sample set. The number of hard sample images in the hard sample set; For the predicted category of sample image i, the true category is... Descending order ranking among all classes The predicted class of sample image j is the true class. Descending order ranking among all classes; This is a sign function, taking the value 0 or 1, if the sample image The true category equals , Take 1, otherwise Set to 0; This is a sign function, taking the value 0 or 1. If the true class of sample image j is equal to... , Take 1, otherwise Take 0.

6. The method for constructing and training an image classification model based on the association learning of easy and difficult samples according to claim 1, characterized in that, After each round of optimization training, the currently optimized image classification model is used to infer the difficult sample images in the current difficult sample set and the easy sample images in the current easy sample set. The current difficult sample set and the current easy sample set are updated based on the inference results for the difficult and easy sample images. After multiple rounds of optimization training, the final image classification model is obtained, which also includes: For a difficult sample image in the current difficult sample set or an easy sample image in the current easy sample set, if the confidence level of the image classification model after current optimization training predicts it as the true class is not less than 0.5, then the sample image is determined to be an easy sample image; otherwise, it is determined to be a difficult sample image. Thus, the current difficult sample set and the current easy sample set are updated, and a new round of optimization training is performed. The optimization is iterated until the image classification model after current optimization training meets the target.

7. The method for constructing and training an image classification model based on the association learning of easy and difficult samples according to claim 1, characterized in that, The easy sample features in the computation feature space Features of difficult samples Pairwise similarity It also includes: Cosine similarity was chosen as an easy sample feature in the feature space. Features of difficult samples The similarity measure function between pairs of objects is calculated using the following formula: ， Among them, similarity A value close to 1 indicates an easy sample feature. Features of difficult samples Similarity; similarity A value close to -1 indicates an easily sampled feature. Features of difficult samples on the contrary.

Images