Image classification model training method, electronic device and computer-readable storage medium

Abstract

The present application discloses an image classification model training method, electronic device and computer-readable storage medium. The training method includes: inputting a training image set and a transformed image set into an image classification model to obtain an original probability matrix and a transformed probability matrix; based on the original probability matrix and a probability threshold, obtaining a classification set corresponding to each image type; each classification set includes at most a second numerical value of training images whose probability exceeds the probability threshold, and the second numerical value is positively correlated with the number of training times of the image classification model; based on each classification set and the transformed image set, obtaining a target matrix; the target matrix includes a judgment result of whether each two transformed images belong to the same image type; based on the target matrix and the transformed probability matrix, obtaining a loss value, adjusting the parameters of the image classification model based on the loss value and updating the current number of training times until a preset convergence condition is met. The above scheme can reduce the difficulty of training and improve the accuracy of image classification.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to an image classification model training method, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the rise of computer vision, image classification, as an important branch of the field, has received increasing attention for its training and application to improve the efficiency of image classification. Using trained image classification models for image classification can effectively improve classification efficiency. However, existing image classification models typically rely on whether there is sufficient labeled data corresponding to the training images, which determines whether to use one of the following training methods: unsupervised training, semi-supervised training, or self-supervised training. This not only places specific requirements on the labeling of training images, but also increases the difficulty of training. When the training method does not match the application scenario, the accuracy of image classification performed by the trained image classification model is also poor. In view of this, how to reduce the training difficulty and improve the accuracy of image classification has become an urgent problem to be solved. Summary of the Invention

[0003] The main technical problem solved by this application is to provide an image classification model training method, electronic device and computer-readable storage medium, which can reduce the training difficulty and improve the accuracy of image classification.

[0004] To solve the above technical problems, the first aspect of the present application provides an image classification model training method, comprising: inputting a training image set and a transformed image set corresponding to the training image set into an image classification model to obtain an original probability matrix and a transformed probability matrix; wherein the training image set includes a first number of image types, the original probability matrix includes the probability of each training image in the training image set belonging to each image type, and the transformed probability matrix includes the probability of each transformed image in the transformed image set belonging to each image type; based on the original probability matrix and a probability threshold, obtaining a classification set corresponding to each of the image types; each of the classification sets includes at most a second number of training images whose probability exceeds the probability threshold, and the second number is positively correlated with the number of training times of the image classification model; based on each of the classification sets and the transformed image set, obtaining a target matrix; the target matrix includes a judgment result of whether each two transformed images belong to the same image type; based on the target matrix and the transformed probability matrix, obtaining a loss value, adjusting the parameters of the image classification model based on the loss value and updating the current number of training times until a preset convergence condition is met, thereby obtaining the trained image classification model.

[0005] To solve the above technical problems, the second aspect of the present application provides an electronic device, which includes: a memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method described in the first aspect above.

[0006] In order to solve the above technical problems, the third aspect of the present application provides a computer-readable storage medium on which program data is stored. When the program data is executed by a processor, the method described in the first aspect is implemented.

[0007] In the above scheme, the training image set and the transformed image set are input into the image classification model, so that the image classification model predicts the probability of each training image in the training image set belonging to each image type, and the probability of each transformed image in the transformed image set belonging to each image type, and obtains the original probability matrix and the transformed probability matrix. Based on the original probability matrix and the probability threshold, the classification set corresponding to each image type is obtained. That is, each image type corresponds to a classification set, and the corresponding training images in the same classification set belong to the same image type. During the iterative process, as the number of training times increases, the second numerical value positively correlated therewith also increases. Therefore, after the number of training times increases, more training images with probabilities exceeding the probability threshold are selected and added to the corresponding classification set, so that the results in the classification set are more reliable. , the higher the accuracy of image classification performed by the trained image classification model, wherein the transformed image corresponds to the training image, and all transformed images in the transformed image set obtained by transforming the same training image belong to the same image type as the corresponding training image. Based on each classification set and the transformed image set, it is possible to judge whether every two transformed images in the transformed image set belong to the same image type, and use the judgment result as the element in the target matrix, and the target matrix can be obtained. The whole process of adjusting the parameters of the image classification model after obtaining the loss value based on the target matrix and the transformation probability matrix can be compatible with any training method of unsupervised training, semi-supervised training and self-supervised training, thereby effectively reducing the difficulty of training and improving the accuracy of image classification using the trained image classification model in different application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without inventive efforts. Among them:

[0009] Figure 1 This is a flowchart of an implementation method of the image classification model training method of the present application;

[0010] Figure 2 This is a flowchart of another embodiment of the image classification model training method of the present application;

[0011] Figure 3 This is a flowchart of another embodiment of the image classification model training method of the present application;

[0012] Figure 4 This is a schematic diagram of an application scenario of an embodiment of the image classification model training method of the present application;

[0013] Figure 5 This is a schematic structural diagram of an embodiment of the electronic device of the present application;

[0014] Figure 6 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0015] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0016] The terms "system" and "network" are often used interchangeably in this document. The term "and / or" is simply a description of an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " generally indicates that the related objects are in an "or" relationship. Furthermore, "multiple" in this document means two or more than two.

[0017] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of the image classification model training method of the present application, which includes:

[0018] S101: Input the training image set and the transformed image set corresponding to the training image set into the image classification model to obtain the original probability matrix and the transformed probability matrix.

[0019] Specifically, the training image set includes a first number of image types, the original probability matrix includes the probability that each training image in the training image set belongs to each image type, and the transformed probability matrix includes the probability that each transformed image in the transformed image set belongs to each image type. The training image set includes multiple training images, and the transformed images in the transformed image set are obtained by transforming the training images. Therefore, all transformed images transformed based on the same training image belong to the same image type as the corresponding training image. If the training image set corresponds to the first number of image types, then the transformed image set also corresponds to the first number of image types.

[0020] In one application method, a training image set and a transformed image set are input into an image classification model, wherein at least some of the training images correspond to labeled data, and the labeled data include the image type to which the training images belong. The image classification model predicts the probability of each training image belonging to each image type based on the labeled data and the image data of the training images, and uses the probability of each training image belonging to each image type as an element in an original probability matrix to obtain an original probability matrix. The image classification model predicts the probability of each transformed image in the transformed image set belonging to each image type based on the labeled data and the transformation method, and uses the probability of each transformed image belonging to each image type as an element in the transformation probability matrix to obtain a transformation probability matrix.

[0021] In another application method, the training image set and the transformed image set are input into an image classification model, the image classification model performs feature extraction on the training images and the transformed images to obtain training image features and transformed image features, the probability of each training image belonging to each image type is predicted based on the training image features, the probability of each training image belonging to each image type is used as an element in the original probability matrix to obtain the original probability matrix, the probability of each transformed image in the transformed image set belonging to each image type is predicted based on the transformed image features and the transformation method, the probability of each transformed image belonging to each image type is used as an element in the transformation probability matrix to obtain the transformation probability matrix.

[0022] In an application scenario, a training image set consisting of multiple training images is obtained, and each training image is randomly transformed multiple times in turn to obtain a transformed image set corresponding to the training image set. Assume that there are B training images in the training image set, and the training image set corresponds to a first numerical image type, the first numerical value is C, and each training image undergoes M random transformations to obtain MB transformed images to form a transformed image set. The training image set and the transformed image set are input into an image classification model so that the image classification model predicts the probability of each training image in the training image set belonging to each image type, and the probability of each transformed image in the transformed image set belonging to each image type, and obtains the original probability matrix P0∈R B×C, transformation probability matrix P∈R MB×C , where the dimension of the original probability matrix P0 is B×C, and the dimension of the transformed probability matrix P is MB×C.

[0023] S102: Based on the original probability matrix and the probability threshold, a classification set corresponding to each image type is obtained, each classification set includes at most a second value of training images whose probability exceeds the probability threshold, and the second value is positively correlated with the number of training times of the image classification model.

[0024] Specifically, the probability threshold is used to filter the probabilities in the original probability matrix, and the probabilities greater than the probability threshold are screened out. The image type pointed to by the screened probabilities is the image type corresponding to the training image. The training image is added to the classification set of the corresponding image type to obtain the classification set corresponding to each image type.

[0025] In one application, the probability threshold is used to filter the probabilities in the original probability matrix row by row to obtain the filtered probabilities that exceed the probability threshold, and the training images and image types corresponding to the filtered probabilities are determined. The training images are then added to the classification set of the corresponding image types to obtain the classification set corresponding to each image type.

[0026] In another application method, the probability threshold is used to filter the probabilities in the original probability matrix column by column to obtain the filtered probabilities that exceed the probability threshold, and the training images and image types corresponding to the filtered probabilities are determined. The training images are then added to the classification set of the corresponding image types to obtain the classification set corresponding to each image type.

[0027] Furthermore, a second value is determined based on the current number of training times. The second value is positively correlated with the number of training times. Each image type corresponds to a classification set. The corresponding training images in the same classification set belong to the same image type. During the iteration process, as the number of training times increases, the second value positively correlated therewith also increases. Therefore, after the number of training times increases, more training images with a probability exceeding the probability threshold are selected and added to the corresponding classification set, so that the results in the classification set are more reliable, and the accuracy of image classification by the trained image classification model is also higher.

[0028] In one application, the second value is positively correlated with the number of training times. For each image type, the first second value training images with the largest probability are taken from the classification set. For classification sets with less than the second value, the training images in them are retained. Furthermore, each image type corresponds to a classification set that includes at most the second value training images whose probability exceeds the probability threshold.

[0029] In a specific application scenario, the ratio of the number of training images to the first value corresponding to the image type is calculated to obtain a fixed ratio, a direct proportional function is generated based on the number of training times, and a second value is determined based on the fixed ratio and the direct proportional function, so that the second value is linearly related to the number of training times, and the second value increases with the increase in the number of training times. Then, in the process of iterative training, as the number of training times increases, the number of training images in the classification set corresponding to each image type increases, and a classification set with gradient changes is obtained, so as to obtain better training effects based on the classification results of gradient changes and improve the training accuracy of the image classification model.

[0030] S103: Obtaining a target matrix based on each classification set and the transformed image set, wherein the target matrix includes a judgment result of whether every two transformed images belong to the same image type.

[0031] Specifically, the transformed images in the transformed image set are obtained by changing the training images. Therefore, all transformed images obtained by transforming the same training image belong to the same image type as the corresponding training image. Therefore, the transformed image set corresponds to reference information that the transformed images based on the same training image belong to the same image type, and the training images in each classification set belong to the same image type.

[0032] In one application, based on reference information corresponding to each classification set and the transformed image set, it is determined whether every two transformed images in the transformed image set belong to the same image type, and the determination result is used as an element in the target matrix to obtain the target matrix.

[0033] In another application method, the number of training times corresponds to a minimum number of training times. When the number of training times does not reach the minimum number of training times, it is determined that the accuracy of the classification set is not high, and then only based on the reference information corresponding to the transformation image set, it is judged whether every two transformed images in the transformation image set belong to the same image type. When the number of training times reaches the minimum number of training times, based on the reference information corresponding to each classification set and the transformation image set, it is judged whether every two transformed images in the transformation image set belong to the same image type, and the judgment results are used as elements in the target matrix to obtain the target matrix.

[0034] In an application scenario, M×B transformed images form a transformed image set. Based on the reference information corresponding to each classification set and the transformed image set, it is judged whether every two transformed images in the transformed image set belong to the same image type. 0 is used to indicate that they do not belong to the same image type, and 1 is used to indicate that they belong to the same image type. The target matrix T with the dimension of MB×MB is obtained, where T ij If it is 1, it means that the transformed image i and the transformed image j belong to the same image type. ij A value of 0 indicates that transformed image i and transformed image j do not belong to the same image type.

[0035] S104: Based on the target matrix and the transformation probability matrix, a loss value is obtained, and the parameters of the image classification model are adjusted based on the loss value and the current number of training times is updated until a preset convergence condition is met to obtain a trained image classification model.

[0036] Specifically, the target matrix and the transformation probability matrix are fed into the loss function layer to obtain a loss value. Based on the loss value, the parameters of the image classification model are adjusted and the current training cycle is updated. The image classification model is repeatedly trained until the preset convergence conditions are met, and the trained image classification model is obtained. Therefore, the loss value can be obtained through unsupervised training, reducing the training difficulty of the image classification model.

[0037] In one application method, the transformed images judged to be of the same image type are determined based on the target matrix, and the image type with the highest probability corresponding to the transformed images is determined based on the probability transformation matrix, that is, the image type corresponding to each transformed image is obtained, and based on the image type corresponding to each transformed image and the transformed images judged to be of the same image type, the loss calculation is performed on the judgment result of the target matrix to obtain the loss value.

[0038] In another application, the transformed images judged to be of the same image type are determined based on the target matrix, the probabilities corresponding to the transformed images judged to be of the same type are extracted from the probability transformation matrix, and the probability distribution similarity is calculated, and the loss value is obtained based on the probability distribution similarity.

[0039] Furthermore, the parameters of the image classification model are adjusted based on the loss value, and the current number of training times is updated after the parameters are adjusted until the preset convergence conditions are met, thereby obtaining a trained image classification model.

[0040] In one application scenario, the preset convergence condition is determined based on the loss value and the confidence of the target matrix. When the loss value is less than the loss threshold and the confidence exceeds the confidence threshold, the training process is ended and the trained image classification model is obtained to obtain an image classification model with higher confidence.

[0041] In another application method, the preset convergence condition is determined based on the loss value and the number of training times. When the loss value is less than the loss threshold and the number of training times exceeds the threshold, the training process is ended to obtain the trained image classification model to obtain an image classification model with higher stability.

[0042] In the above scheme, the training image set and the transformed image set are input into the image classification model, so that the image classification model predicts the probability of each training image in the training image set belonging to each image type, and the probability of each transformed image in the transformed image set belonging to each image type, and obtains the original probability matrix and the transformed probability matrix. Based on the original probability matrix and the probability threshold, the classification set corresponding to each image type is obtained. That is, each image type corresponds to a classification set, and the corresponding training images in the same classification set belong to the same image type. During the iterative process, as the number of training times increases, the second numerical value positively correlated therewith also increases. Therefore, after the number of training times increases, more training images with probabilities exceeding the probability threshold are selected and added to the corresponding classification set, so that the results in the classification set are more reliable. , the higher the accuracy of image classification performed by the trained image classification model, wherein the transformed image corresponds to the training image, and all transformed images in the transformed image set obtained by transforming the same training image belong to the same image type as the corresponding training image. Based on each classification set and the transformed image set, it is possible to judge whether every two transformed images in the transformed image set belong to the same image type, and use the judgment result as the element in the target matrix, and the target matrix can be obtained. The whole process of adjusting the parameters of the image classification model after obtaining the loss value based on the target matrix and the transformation probability matrix can be compatible with any training method of unsupervised training, semi-supervised training and self-supervised training, thereby effectively reducing the difficulty of training and improving the accuracy of image classification using the trained image classification model in different application scenarios.

[0043] See also Figure 2 , Figure 2 This is a flow chart of another embodiment of the image classification model training method of the present application, which includes:

[0044] S201: Inputting a training image set and a transformed image set corresponding to the training image set into an image classification model to obtain an original probability matrix and a transformed probability matrix.

[0045] Specifically, a plurality of training images constitute a training image set, and a transformed image is obtained after random transformation of the training image. The training image set and the transformed image set are input into an image classification model to obtain an original probability matrix and a transformed probability matrix. The training image set includes a first number of image types. The original probability matrix includes the probability that each training image in the training image set belongs to each image type, and the transformed probability matrix includes the probability that each transformed image in the transformed image set belongs to each image type.

[0046] In one application method, before inputting a training image set and a transformed image set corresponding to the training image set into an image classification model to obtain an original probability matrix and a transformed probability matrix, the method further includes: obtaining a training image set consisting of multiple training images, randomly transforming each training image in the training image set, and obtaining a transformed image set corresponding to the training image set; and determining all transformed images in the transformed image set that are transformed based on the same training image as belonging to the same image type, and obtaining prior information corresponding to the transformed image set.

[0047] Specifically, a training image set consisting of multiple training images is obtained, and each training image is randomly transformed the same number of times to obtain a transformed image set consisting of multiple transformed images, wherein all transformed images obtained by transforming the same training image belong to the same image type as the corresponding training image, and all transformed images in the transformed image set that are transformed based on the same training image are determined to belong to the same image type, thereby obtaining prior information corresponding to the transformed image set.

[0048] Furthermore, the prior information corresponding to the transformed image set can serve as an important basis for subsequently obtaining the target matrix and adjusting the image classification model.

[0049] S202: Filter the probabilities in the original probability matrix using the probability threshold to obtain filtered probabilities that exceed the probability threshold.

[0050] Specifically, the probability threshold is used to filter out probabilities with higher confidence in the original probability matrix, thereby obtaining filtered probabilities that exceed the probability threshold.

[0051] In an application scenario, there are B training images in the training image set, and the training image set corresponds to the first value image type, the first value is C, and the original probability matrix P0∈R B×C , where each row has C elements, corresponding to the probability that the training image in the i-th row belongs to each image type, where the value corresponding to the probability is between 0 and 1 and includes 0 and 1. The probability threshold is used to filter the probability in the original probability matrix to obtain the filtered probability.

[0052] In one specific application scenario, the probability threshold approaches 1, for example, 0.9999. The maximum value is taken row by row in the original probability matrix, that is, the image type with the highest probability corresponding to each training image. The probability threshold is used to filter the maximum value in each row to obtain a high-confidence filtering probability. It should be noted that in other application scenarios, the probability threshold can be any value between 0.9 and 1, and this application does not impose specific restrictions on this.

[0053] S203: Based on the image type corresponding to the filtering probability and the position index of the training image corresponding to the filtering probability, an initial set corresponding to each image type is obtained.

[0054] Specifically, in the original probability matrix, the filter probabilities correspond to image types and training images. The position index of the training image corresponding to the filter probability is extracted and added to the initial set of image types corresponding to the filter probability, resulting in an initial set corresponding to each image type. The corresponding training image can be found by the position index, and the position index is used as an element in the initial set to generate the initial set.

[0055] S204: Obtain a second value based on the current number of training times, the first value, and the number of training images.

[0056] Specifically, the second value is determined by comprehensively utilizing the current number of training times, the first value, and the number of training images, so as to obtain the second value that is positively correlated with the number of training times.

[0057] In one application, the number of training times corresponds to a maximum number of training times and a minimum number of training times. Based on the current number of training times, the first value and the number of training images, obtaining the second value includes: obtaining a proportional value based on the maximum number of training times, the minimum number of training times and the current number of training times; the proportional value is positively correlated with the current number of training times; based on the proportional value and the ratio of the number of training images to the first value, obtaining a second value positively correlated with the number of training times.

[0058] Specifically, a proportional function is generated based on the maximum number of training times, the minimum number of training times, and the current number of training times. The current number of training times is substituted into the proportional function to obtain the proportional value. The above process is expressed as follows:

[0059]

[0060] Among them, epoch represents the current number of training times, epoch thres Represents the minimum number of training times, epoch max Represents the maximum number of training times. During the iteration process, as the number of training times increases, the proportional value obtained based on the proportional function increases accordingly. Therefore, the proportional value is positively correlated with the current number of training times.

[0061] Furthermore, a quantity function is constructed based on the number of training images, the first value, and the ratio value. The ratio of the number of training images to the first value is calculated and multiplied by the ratio value. The result of the multiplication corresponds to a fixed coefficient, thereby determining the second value. The above process is expressed by the following formula:

[0062]

[0063] Among them, B represents the number of training images, C represents the first value corresponding to the image type, and 0.3 is a fixed coefficient. Based on the above quantity function, a second value positively correlated with the number of training times can be obtained. The calculation process of the second value combines the number of training images and the number of image types to better adapt to different application scenarios. In different application scenarios, the fixed coefficient can also be customized.

[0064] S205: Based on the filtering probabilities corresponding to the position indexes, select the first second value position indexes with the largest probabilities to obtain a classification set consisting of at most the second value position indexes.

[0065] Specifically, the position indexes in the initial classification set correspond to training images, and the training images correspond to filtering probabilities. When the number of position indexes in any initial set exceeds a second value, the first second value position indexes with the largest probability are selected in the corresponding initial set, so that the classification set corresponding to each image type includes at most the second value position indexes, so as to obtain a relatively balanced and concise classification set.

[0066] S206: Obtain a target matrix based on each classification set and the transformed image set, wherein the target matrix includes a judgment result of whether every two transformed images belong to the same image type.

[0067] Specifically, in response to the current number of training times not reaching the minimum number of training times, based on the prior information corresponding to the transformed images, it is judged whether any two transformed images in the transformed image set belong to the same image type, and a target matrix is ​​obtained. In response to the current number of training times reaching the minimum number of training times, based on the prior information corresponding to each classification set and the transformed images, it is judged whether any two transformed images in the transformed image set belong to the same image type, and a target matrix is ​​obtained.

[0068] That is to say, if the current number of training times does not reach the minimum number of training times, the results of determining the original probability matrix and the classification set are not accurate enough. Therefore, based only on the prior information corresponding to the transformed images, whether any two transformed images in the transformed image set belong to the same image type is determined to obtain the target matrix. When the number of training times reaches the minimum number of training times, it is determined that the original probability matrix and the classification set are relatively reliable after being optimized for at least the minimum number of training times. Then, based on the prior information corresponding to each classification set and the transformed image, whether any two transformed images in the transformed image set belong to the same image type is determined to obtain the target matrix.

[0069] Furthermore, the prior information includes all transformed images in the transformed image set that are transformed based on the same training image, which are determined to belong to the same image type, and the training images corresponding to the position index in the same classification set belong to the same image type. Therefore, when the accuracy of the classification set is high and the classification result is relatively reliable, based on the prior information corresponding to each classification set and the transformed image, it is judged whether any two transformed images in the transformed image set belong to the same image type. Using the judgment result as an element in the target matrix can obtain a more accurate target matrix, thereby improving the accuracy of image classification prediction.

[0070] S207: Based on the target matrix and the transformation probability matrix, a loss value is obtained, and the parameters of the image classification model are adjusted based on the loss value and the current number of training times is updated until a preset convergence condition is met to obtain a trained image classification model.

[0071] Specifically, based on the target matrix, the transformed images judged to be of the same image type are determined, the probabilities corresponding to the transformed images judged to be of the same type are extracted from the probability transformation matrix, and the probability distribution similarity is calculated. The loss value is obtained based on the probability distribution similarity. Among them, the target matrix and the transformation probability matrix are both obtained during the training process, so the loss value does not need to be obtained based on accurate labeled data, thereby reducing the difficulty of training.

[0072] Furthermore, the parameters of the image classification model are adjusted based on the loss value and the current number of training times is updated. The image classification model is repeatedly trained until the preset convergence conditions are met to obtain a trained image classification model.

[0073] In this embodiment, an original probability matrix corresponding to a training image set and a transformed probability matrix corresponding to a transformed image set are obtained based on an image classification model. The maximum value of each row of the original probability matrix is ​​filtered using a probability threshold to obtain a high-confidence filtered probability. A second value is determined comprehensively using the current number of training times, the first value, and the number of training images to obtain a second value that is positively correlated with the number of training times. Position indexes corresponding to at most the second value of training images are selected in the classification set to obtain a relatively balanced and streamlined classification set. When the number of training times exceeds the minimum number of training times, based on the prior information corresponding to each classification set and the transformed image, whether any two transformed images in the transformed image set belong to the same image type is judged to obtain a target matrix. Based on the target matrix and the transformed probability matrix, a loss value is obtained. The target matrix and the transformed probability matrix are both obtained during the training process. Therefore, the loss value does not need to be obtained based on accurate labeled data, thereby reducing the difficulty of training.

[0074] See also Figure 3 , Figure 3This is a flow chart of another embodiment of the image classification model training method of the present application. The image classification model includes a feature extraction module and a classification module. The method includes:

[0075] S301: Input the training image set and the transformed image set into a feature extraction module to obtain an original feature matrix and a transformed feature matrix, wherein the original feature matrix includes features corresponding to each training image, and the transformed feature matrix includes features corresponding to each transformed image.

[0076] Specifically, a training image set consisting of a plurality of training images and a transformed image set consisting of a plurality of transformed images obtained by randomly transforming the training images are obtained. The training image set and the transformed image set are input into a feature extraction module, so that the feature extraction module performs feature extraction on the training images and the transformed images, thereby fully extracting features on the images and generating an original feature matrix corresponding to the training image set and a transformed feature matrix corresponding to the transformed image set. The original feature matrix includes features corresponding to each training image, and the transformed feature matrix includes features corresponding to each transformed image.

[0077] In one application, the feature extraction module includes multiple cascaded convolutional neural networks for extracting features from images. The training image set and the transformed image set are input into the feature extraction module to obtain features corresponding to the images, so that the amount of feature data is greatly reduced compared to the original image, and the significant features of the image are obtained to facilitate image classification. The dimension of the feature is related to the number of channels of the convolutional neural network.

[0078] S302: Input the original feature matrix and the transformed feature matrix into a classification module to obtain an original probability matrix and a transformed probability matrix.

[0079] Specifically, the original feature matrix and the transformed feature matrix are input into the classification module, so that the classification module predicts the image type of the training image based on the features corresponding to the training image, and predicts the image type of the transformed image based on the features corresponding to the transformed image, thereby obtaining the probability of each training image belonging to each image type and the probability of each transformed image belonging to each image type, thereby generating the original probability matrix and the transformed probability matrix. The original probability matrix includes the probability of each training image in the training image set belonging to each image type, and the transformed probability matrix includes the probability of each transformed image in the transformed image set belonging to each image type.

[0080] In one application, the classification module includes a cascaded perception network and a classification head, which is used to classify the image according to the features corresponding to the image and obtain the probability that the input image belongs to each image type.

[0081] S303: Based on the original probability matrix and the probability threshold, a classification set corresponding to each image type is obtained.

[0082] Specifically, the probabilities in the original probability matrix are filtered using a probability threshold to select probabilities greater than the probability threshold. The image types pointed to by the selected probabilities are the image types corresponding to the training images. The training images are then added to the classification sets of the corresponding image types to obtain classification sets corresponding to each image type. Each classification set includes at most a second value of training images with probabilities exceeding the probability threshold, and the second value is positively correlated with the number of training times of the image classification model.

[0083] Furthermore, the elements included in each classification set are position indexes corresponding to the training images. The corresponding training images can be found through the position indexes, and the position indexes are used as elements in the classification set. Each classification set includes position indexes corresponding to at most the second number of training images.

[0084] S304: In the original feature matrix, determine the features corresponding to the training images in each classification set, and calculate the average features corresponding to each classification set.

[0085] Specifically, the classification set corresponds to a position index corresponding to the training image, and the features corresponding to the training image are determined in the original feature matrix based on the position index, thereby obtaining the features corresponding to the training image in each classification set.

[0086] Furthermore, the features corresponding to the training images in each classification set are averaged to obtain the feature center corresponding to each classification set.

[0087] S305: Based on each classification set and its corresponding average feature, obtain the top third numerical value of training images with the highest similarity to the average feature and the probability exceeding the probability threshold, so as to update each classification set, and the third numerical value is less than the second numerical value.

[0088] Specifically, the similarity between the features and the feature centers is determined in each separation set, and the training images with the top third numerical values ​​of the highest similarity to the average feature exceeding the probability threshold are obtained, thereby updating each classification set, where the third numerical value is less than the second numerical value.

[0089] In one application method, the following steps are performed on each classification set: the features corresponding to the classification set are determined based on the position index in the classification set, the feature average corresponding to all features in the classification set is calculated, the feature average is used as the feature center point, the cosine distance between each feature in the classification set and the feature center is calculated, the similarity between the feature and the feature center is fed back based on the cosine distance, and the classification set is updated using the position index corresponding to the third training image with the largest cosine distance value.

[0090] In another application method, the following steps are performed on each classification set: the features corresponding to the classification set are determined based on the position index in the classification set, the feature average corresponding to all features in the classification set is calculated, the similarity between each feature in the classification set and the average feature is scored using a similarity algorithm, and the classification set is updated using the position index corresponding to the third training image with the largest score.

[0091] It is understandable that when the second value is greater than 1, the third value is less than the second value, so as to further filter the position indexes corresponding to the training images in each classification set and improve the accuracy of image classification.

[0092] S306: Obtain a target matrix based on each classification set and the transformed image set, wherein the target matrix includes a judgment result of whether each two transformed images belong to the same image type.

[0093] Specifically, how to construct the target matrix can be referred to any of the above embodiments, and this application will not elaborate on this.

[0094] S307: Based on the transformation feature matrix, obtain a feature similarity matrix corresponding to the transformation image set; based on the transformation probability matrix, obtain a probability similarity matrix corresponding to the transformation image set.

[0095] Specifically, the similarity between the features corresponding to each two transformed images is determined in the transformation feature matrix to obtain a feature similarity matrix corresponding to the transformed image set. The similarity between the probabilities corresponding to each two transformed images is determined in the transformation probability matrix to obtain a probability similarity matrix corresponding to the transformed image set. Each element in the feature similarity matrix includes the similarity between the features of each two transformed images, and each element in the probability similarity matrix includes the similarity between the probabilities corresponding to each two transformed images.

[0096] In one application scenario, the similarity between the features corresponding to each two transformed images is determined based on the cosine distance, so that the similarity between the features can be obtained through simple calculations, thereby improving the efficiency of generating the feature similarity matrix. The above process is expressed by the following formula:

[0097] S feat ij =cos(z i ,z j ) (3)

[0098] Among them, S feat ∈R MB×MB , represents the similarity between the features corresponding to each two transformed images, and the cosine distance is used to feedback the similarity between two features.

[0099] In one application scenario, the similarity between the probabilities corresponding to each two transformed images is determined based on the entropy function to generate a probability similarity matrix. The above process is expressed as follows:

[0100]

[0101] Among them, S prob ∈R MB×MB , represents the similarity between the probabilities corresponding to each two transformed images, H() represents the entropy function, and based on the information entropy between the features, the divergence value (Jensen-Shannon divergence) between the probabilities corresponding to each two transformed images is determined, referred to as JS divergence, where JS divergence represents the similarity between two probability distributions to improve the accuracy of obtaining the similarity between probabilities.

[0102] S308: Obtain a loss value based on the target matrix, the feature similarity matrix, and the probability similarity matrix.

[0103] Specifically, based on the target matrix, the two transformed images with the same image type are searched and judged. In the feature similarity matrix and the probability similarity matrix, the feature loss and probability loss between the two transformed images of the same image type are determined respectively. The feature loss and probability loss are weighted and summed to obtain the loss value of this training.

[0104] In one application method, the judgment results in the target matrix include true values ​​and false values, and the loss value is obtained based on the target matrix, the feature similarity matrix and the probability similarity matrix, including: based on the true value in the target matrix, extracting the feature true value similarity corresponding to the true value in the feature similarity matrix, and extracting the probability true value similarity corresponding to the true value in the probability similarity matrix; performing a conversion operation on all feature true value similarities to obtain feature loss, and performing a conversion operation on all probability true value similarities to obtain probability loss; the conversion operation includes calculating the logarithm of the sum of the exponentials; and performing a weighted summation on the feature loss and the probability loss to obtain the loss value.

[0105] Specifically, based on the true value in the target matrix, two transformed images belonging to the same image type are determined, and the feature true value similarity corresponding to the true value is extracted from the feature similarity matrix, that is, the feature similarity between the two transformed images of the same image type is determined, and the probability true value similarity corresponding to the true value is extracted from the probability similarity matrix, that is, the probability similarity between the two transformed images of the same image type is determined, and the feature true value similarity and the probability true value similarity are converted. The conversion operation includes calculating the logarithm of the sum of exponentials (LogSumExp, LSE), that is, taking the logarithm of the sum of exponentials of the input parameters, thereby reducing the overflow and underflow of the probability distribution when obtaining the similarity of the probability distribution, and obtaining the feature loss and probability loss respectively. The above process is expressed as follows:

[0106]

[0107]

[0108] Among them, there are B training images in the training image set, each training image undergoes M random transformations, and MB transformed images are obtained to form the transformed image set, class = prob, feat, that is, the feature similarity matrix and the probability similarity matrix distribution are calculated once to obtain the feature loss l feat and probability loss l prob , lse() represents the conversion operation, where T ij A value of 1 indicates that the transformed image i and the transformed image j belong to the same image type, where j≠i means avoiding comparison between the transformed image and itself.

[0109] Furthermore, the probability loss and feature loss are weighted and summed to obtain the loss value. The above process is expressed as follows:

[0110] l=a*l prob +b*l feat ) (7)

[0111] Among them, a and b represent weighting coefficients. In some application scenarios, a and b are both 1 / 2. In other application scenarios, they can also be two values ​​less than 0-1 whose sum is 1. In this way, the training loss value is determined based on the two dimensions of features and probability, thereby improving the training accuracy of the image classification model.

[0112] S309: Adjust the parameters of the image classification model based on the loss value and update the current number of training times until the preset convergence conditions are met to obtain the trained image classification model.

[0113] Specifically, the parameters of the image classification model are adjusted based on the loss value and the current number of training cycles is updated. The image classification model is repeatedly trained until the preset convergence conditions are met, thereby obtaining a trained image classification model. Therefore, the loss value can be obtained through unsupervised training to reduce the difficulty of training the image classification model.

[0114] The trained image classification model is then tested. During the testing phase, if the number of image types is predetermined, the classification labels are obtained by taking the index corresponding to the maximum value in each row of the target matrix output by the image classification model. If the number of image types is not predetermined, the features output by the image classification model are extracted as image features, and clustered using a clustering method, including but not limited to the density-based spatial clustering of applications with noise (DBSCAN), to obtain the classification labels.

[0115] In a specific application scenario, please refer to Figure 4 , Figure 4 This is a schematic diagram of an application scenario of an embodiment of the image classification model training method of the present application. The image classification model 40 includes a feature extraction module 400 and a classification module 402, wherein the feature extraction module 400 specifically includes multiple cascaded residual neural networks (Residual Neural Network), and the classification module 402 includes two layers of multilayer perceptrons (Multilayer Perceptron) and a fully connected layer (Fully Connected Layer). Each training image in a set of B training images is randomly transformed M times to obtain a transformed image set, wherein B is 512. The training image set and the transformed image set are input into the feature extraction module 400 to obtain the original feature matrix and the transformed feature matrix, and the original feature matrix and the transformed feature matrix are input into the classification module 402 to obtain the original probability matrix and the transformed probability matrix. A classification set corresponding to each image type is constructed based on the original probability matrix and the original feature matrix. The position index corresponding to the training image in the classification set is truncated using a high-confidence probability threshold, and the position index in the classification set is step-by-step screened based on a second numerical value that is positively correlated with the number of training times to obtain a classification set with gradient changes. The loss value is obtained based on the target matrix, the feature similarity matrix and the probability similarity matrix, and the Adam optimizer is used for training so that it is suitable for training data with gradient changes.

[0116] In this embodiment, a classification set is constructed based on the original probability matrix to obtain a classification set including position indices corresponding to at most a second numerical value of training images, and then the classification set is further filtered based on the original feature matrix to obtain a classification set including position indices corresponding to at most a third numerical value of training images, thereby improving the accuracy of the classification set, and determining the loss value of training based on the two dimensions of features and probability, thereby improving the training accuracy of the image classification model and the accuracy of image classification performed by the trained image classification model.

[0117] See also Figure 5 , Figure 5 This is a structural diagram of an embodiment of an electronic device of the present application. The electronic device 50 includes a memory 501 and a processor 502 coupled to each other, wherein the memory 501 stores program data (not shown in the figure), and the processor 502 calls the program data to implement the method in any of the above embodiments. For an explanation of the relevant content, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0118] See also Figure 6 , Figure 6 This is a structural diagram of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 60 stores program data 600. When the program data 600 is executed by the processor, the method in any of the above embodiments is implemented. For an explanation of the relevant content, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0119] It should be noted that the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of this embodiment.

[0120] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0121] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0122] The above description is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for training an image classification model, characterized in that: The method comprises: Inputting a training image set and a transformed image set corresponding to the training image set into an image classification model to obtain an original probability matrix and a transformed probability matrix; wherein the training image set includes a first number of image types, the original probability matrix includes the probability of each training image in the training image set belonging to each image type, and the transformed probability matrix includes the probability of each transformed image in the transformed image set belonging to each image type; Based on the original probability matrix and the probability threshold, a classification set corresponding to each of the image types is obtained; each of the classification sets includes at most a second value of training images whose probability exceeds the probability threshold, and the second value is positively correlated with the number of training times of the image classification model; Based on each of the classification sets and the transformed image set, a target matrix is ​​obtained; the target matrix includes a determination result of whether each two transformed images belong to the same image type; Based on the target matrix and the transformation probability matrix, a loss value is obtained, and based on the loss value, the parameters of the image classification model are adjusted and the current number of training times is updated until a preset convergence condition is met, thereby obtaining the trained image classification model.

2. The image classification model training method according to claim 1, characterized in that: The obtaining, based on the original probability matrix and the probability threshold, a classification set corresponding to each image type, includes: Filtering the probabilities in the original probability matrix using the probability threshold to obtain filtered probabilities that exceed the probability threshold; Based on the image type corresponding to the filtering probability and the position index of the training image corresponding to the filtering probability, obtaining an initial set corresponding to each of the image types; Obtaining the second value based on the current number of training times, the first value, and the number of the training images; Based on the filtering probabilities corresponding to the position indexes, the second value of position indexes with the largest probabilities are selected to obtain the classification set consisting of at most the second value of position indexes.

3. The image classification model training method according to claim 2, characterized in that: The number of training times corresponds to a maximum number of training times and a minimum number of training times, and obtaining the second value based on the current number of training times, the first value, and the number of training images includes: Obtaining a ratio value based on the maximum number of training times, the minimum number of training times, and the current number of training times; wherein the ratio value is positively correlated with the current number of training times; Based on the proportion value and the ratio of the number of the training images to the first value, the second value positively correlated with the number of training times is obtained.

4. The image classification model training method according to claim 3, characterized in that: Before inputting the training image set and the transformed image set corresponding to the training image set into the image classification model to obtain the original probability matrix and the transformed probability matrix, the method further includes: Obtaining a training image set consisting of a plurality of training images, and performing random transformation on each of the training images in the training image set to obtain the transformed image set corresponding to the training image set; determining all transformed images in the transformed image set that are transformed based on the same training image as belonging to the same image type, and obtaining prior information corresponding to the transformed image set; The step of obtaining a target matrix based on each of the classification sets and the transformed image set includes: In response to the current number of training times not reaching the minimum number of training times, judging whether any two transformed images in the transformed image set belong to the same image type based on prior information corresponding to the transformed images, to obtain the target matrix; In response to the current number of training times reaching the minimum number of training times, based on the prior information corresponding to each of the classification sets and the transformed images, whether any two transformed images in the transformed image set belong to the same image type is judged to obtain the target matrix.

5. The image classification model training method according to claim 1, characterized in that: The image classification model includes a feature extraction module and a classification module. The training image set and the transformed image set corresponding to the training image set are input into the image classification model to obtain the original probability matrix and the transformed probability matrix, including: Inputting the training image set and the transformed image set into the feature extraction module to obtain an original feature matrix and a transformed feature matrix; wherein the original feature matrix includes features corresponding to each of the training images, and the transformed feature matrix includes features corresponding to each of the transformed images; The original feature matrix and the transformed feature matrix are input into the classification module to obtain the original probability matrix and the transformed probability matrix.

6. The image classification model training method according to claim 5, characterized in that: After obtaining a classification set corresponding to each of the image types based on the original probability matrix and the probability threshold, the method further includes: In the original feature matrix, determining the features corresponding to the training images in each of the classification sets, and calculating the average features corresponding to each of the classification sets; Based on each of the classification sets and its corresponding average features, the first third numerical value of training images with the highest similarity to the average feature exceeding the probability threshold are obtained to update each of the classification sets; the third numerical value is less than the second numerical value.

7. The image classification model training method according to claim 5, characterized in that: The obtaining of the loss value based on the target matrix and the transformation probability matrix includes: Based on the transformation feature matrix, a feature similarity matrix corresponding to the transformed image set is obtained, and based on the transformation probability matrix, a probability similarity matrix corresponding to the transformed image set is obtained; each element in the feature similarity matrix includes the similarity between the features of each two transformed images, and each element in the probability similarity matrix includes the similarity between the probabilities corresponding to each two transformed images; The loss value is obtained based on the target matrix, the feature similarity matrix, and the probability similarity matrix.

8. The image classification model training method according to claim 7, characterized in that: The judgment result includes a true value and a false value, and obtaining the loss value based on the target matrix, the feature similarity matrix, and the probability similarity matrix includes: Based on the true value in the target matrix, extracting the feature truth value similarity corresponding to the true value in the feature similarity matrix, and extracting the probability truth value similarity corresponding to the true value in the probability similarity matrix; Performing a conversion operation on all the feature truth value similarities to obtain feature loss, and performing a conversion operation on all the probability truth value similarities to obtain probability loss; the conversion operation includes calculating the logarithm of the sum of the exponentials; A weighted sum is performed on the feature loss and the probability loss to obtain the loss value.

9. An electronic device, characterized in that: include: A memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having program data stored thereon, characterized in that: When the program data is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images