Image classification method, device, storage medium and electronic device

Abstract

The present invention provides an image classification method, apparatus, storage medium, and electronic device. The method comprises: performing multiple rounds of classification on multiple images to be classified, wherein the i-th round of classification includes: obtaining a set of low-confidence images output by an image classification model in the i-th round of classification; inputting the low-confidence image set into the image classification model in the i-th round of classification to obtain a set of high-confidence images output in the i-th round of classification. This invention solves the problem of low efficiency in image type annotation in the prior art, thereby achieving the effect of automatically annotating image types.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to an image classification method, device, storage medium and electronic device. Background Art

[0002] In the field of image recognition, the amount of training image data has a certain impact on the recognition accuracy of the neural network model. That is, the larger the amount of training image data, the higher the recognition accuracy of the neural network model.

[0003] Existing techniques typically use manual methods to annotate training images, such as labeling the types of objects in an image. However, manual annotation of training images is inefficient and consumes a large amount of human resources. This makes it difficult to annotate large numbers of training images.

[0004] Regarding the problem of low efficiency in image type annotation in related technologies, no effective solution has been proposed so far. Summary of the Invention

[0005] Embodiments of the present invention provide an image classification method, device, storage medium, and electronic device to at least solve the problem of low efficiency in image type labeling in related technologies.

[0006] According to one embodiment of the present invention, a method for classifying an image is provided, comprising: performing multiple rounds of classification operations on multiple images to be classified, wherein the i-th round of classification operation comprises: obtaining a low-confidence image set output by an image classification model in an i-1-th round of classification operation, wherein the low-confidence image set includes images whose first confidence determined in the i-1-th round of classification operation is less than or equal to a preset value, and the first confidence is obtained by processing the corresponding images using the image classification model in the i-1-th round of classification operation, where i is greater than or equal to 2; inputting the low-confidence image set into the image classification model in the i-th round of classification operation to obtain a high-confidence image set output in the i-th round of classification operation, wherein the high-confidence image set includes images whose second confidence determined in the i-th round of classification operation is greater than a preset value, and the second confidence is obtained by processing the corresponding images using the image classification model in the i-1-th round of classification operation.

[0007] Optionally, after obtaining the high-confidence image set output in the i-th round of classification operation, the method further includes: obtaining the classification result of the image classification model in the i-th round of classification operation on the high-confidence image set; and determining the image type in the classification result as the type of the image in the high-confidence image set.

[0008] Optionally, before inputting the low-confidence image set into the image classification model in the i-th round of classification operation to obtain the high-confidence image set output in the i-th round of classification operation, the method further includes: acquiring the low-confidence image set and the high-confidence image set output in the i-1-th round of classification operation, wherein the high-confidence image set includes images whose first confidence level determined in the i-1-th round of classification operation is greater than the preset value; and using the low-confidence image set and the high-confidence image set output in the i-1-th round of classification operation to train the image classification model in the i-th round of classification operation.

[0009] Optionally, the use of the low-confidence image set and the high-confidence image set output in the i-1th round of classification operation to train the image classification model in the i-1th round of classification operation includes: using the low-confidence image set output in the i-1th round of classification operation to train the feature network to obtain the feature network in the i-1th round of classification operation; using the high-confidence image set output in the i-1th round of classification operation to train the classification network to obtain the classification network in the i-1th round of classification operation; and determining the feature network in the i-1th round of classification operation and the classification network in the i-1th round of classification operation as the image classification model in the i-1th round of classification operation.

[0010] Optionally, the using the high-confidence image set output in the i-1th round of classification operation to train the classification network to obtain the classification network in the i-th round of classification operation includes: using the high-confidence image set output in the i-1th round of classification operation and a labeled image set to train the classification network to obtain the classification network in the i-th round of classification operation, wherein the labeled image set includes images of known types.

[0011] Optionally, the feature network is trained using the set of low-confidence images output in the i-1th round of classification operation to obtain the feature network in the i-th round of classification operation, including using the set of low-confidence images output in the i-1th round of classification operation to perform multiple iterative training on the feature network to obtain the feature network in the i-th round of classification operation, wherein the output of the feature network in the i-th round of classification operation satisfies a preset loss function, and the target parameter in the preset loss function increases according to a preset adjustment value as the number of iterations increases.

[0012] Optionally, the classification network is trained using the high-confidence image set output in the i-1th round of classification operation to obtain the classification network in the i-th round of classification operation, including: using the high-confidence image set output in the i-1th round of classification operation to perform multiple iterative training on the classification network to obtain the classification network in the i-th round of classification operation, wherein the output value of the loss function between the estimated image type output by the classification network in the i-th round of classification operation and the known image type of the high-confidence image set satisfies a preset convergence condition.

[0013] Optionally, the method further includes: obtaining a feature network in the first round of classification operation by training a set of labeled images and a set of unlabeled images, wherein the set of unlabeled images includes images of unknown types and the set of labeled images includes images of known types; obtaining a classification network in the first round of classification operation by training the set of labeled images; and determining the feature network in the first round of classification operation and the classification network in the first round of classification operation as the image classification model in the first round of classification operation.

[0014] According to another embodiment of the present invention, a device for determining image types is provided, which is used to perform multiple rounds of classification operations on multiple images to be classified, including: an acquisition unit, used to acquire a low-confidence image set output in the i-1th round of classification operation, wherein the low-confidence image set includes images whose first confidence determined in the i-1th round of classification operation is less than or equal to a preset value, and the first confidence is obtained by processing the corresponding images using the image classification model in the i-1th round of classification operation, where i is greater than or equal to 2; an input unit, used to input the low-confidence image set into the image classification model in the i-th round of classification operation to obtain a high-confidence image set output in the i-th round of classification operation, wherein the high-confidence image set includes images whose second confidence determined in the i-1th round of classification operation is greater than a preset value, and the second confidence is obtained by processing the corresponding images using the image classification model in the i-1th round of classification operation.

[0015] According to yet another embodiment of the present invention, a storage medium is provided, in which a computer program is stored. The computer program is configured to execute the steps of any one of the above method embodiments when running.

[0016] According to another embodiment of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any one of the above method embodiments.

[0017] The present invention inputs the low-confidence image set output by the image classification model in the i-1th round of classification operation into the image classification model in the i-th round of classification operation to obtain the high-confidence image set output in the i-th round of classification operation. The image types of the high-confidence image set output by the image classification model in the i-th round of classification operation are used as the types of the high-confidence images. This achieves the purpose of classifying images using the image classification model and achieves the effect of automatically labeling image types. This can also solve the problem of low efficiency in image type labeling caused by manual labeling of image types in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0019] Figure 1 This is a hardware structure block diagram of a mobile terminal for an image classification method according to an embodiment of the present invention;

[0020] Figure 2 is a flow chart of an image classification method according to an embodiment of the present invention;

[0021] Figure 3 is an optional feature network training flow chart based on contrastive learning according to an embodiment of the present invention;

[0022] Figure 4 is an optional supervised classification network training flow chart according to an embodiment of the present invention;

[0023] Figure 5 is a flow chart of automatic classification samples based on contrastive learning according to an optional embodiment of the present invention;

[0024] Figure 6 is a flow chart of classification network training based on contrastive learning according to an optional embodiment of the present invention;

[0025] Figure 7 FIG. 4 is a structural block diagram of an image classification device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that, unless there is a conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0027] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0028] The method embodiment provided in the first embodiment of the present application can be executed in a terminal device (such as a mobile terminal, a computer terminal, or an intelligent camera (IPC)), a server, or a similar computing device. Taking running on a terminal device as an example, Figure 1 FIG. 1 is a hardware structure diagram of a terminal device for a method for determining an image type according to an embodiment of the present invention. Figure 1 As shown, the terminal device may include one or more ( Figure 1 Only one is shown in the figure) processor 102 (processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA and other processing devices) and a memory 104 for storing data. Optionally, the terminal device may also include a transmission device 106 and an input / output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0029] Memory 104 can be used to store computer programs, such as software programs and modules of application software, such as the computer program corresponding to the method for determining the image type in the embodiments of the present invention. Processor 102 executes the computer programs stored in memory 104 to execute various functional applications and data processing, thereby implementing the aforementioned methods. Memory 104 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 104 may further include memory remotely located relative to processor 102, and such remote memory may be connected to a terminal device via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0030] The transmission device 106 is used to receive or send data via a network. A specific example of the aforementioned network may include a wireless network provided by a telecommunications provider of the terminal device. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0031] In this embodiment, a method for determining the image type running on the above terminal device is provided. Figure 2 is a flow chart of an image classification method according to an embodiment of the present invention, such as Figure 2 As shown, the process includes the following steps:

[0032] Step S202: performing multiple rounds of classification operations on the multiple images to be classified, wherein the i-th round of classification operation includes: obtaining a low-confidence image set output by the image classification model in the i-1-th round of classification operation, wherein the low-confidence image set includes images having a first confidence level determined in the i-1-th round of classification operation that is less than or equal to a preset value, the first confidence level being obtained by processing the corresponding images using the image classification model in the i-1-th round of classification operation, and i being greater than or equal to 2;

[0033] In which, the multiple images to be classified can be image samples of unknown types, and multiple rounds of classification operations are performed on the multiple images to be classified. The low-confidence image set output by the image classification model in the i-1th round of classification operation includes images whose first confidence is less than or equal to a preset value, and the first confidence is obtained after processing multiple images of unknown types in the image classification model in the i-1th round of classification operation.

[0034] In the embodiments of this application, confidence is also referred to as reliability or confidence level. In this embodiment, confidence can be used to indicate the probability that an image output by an image classification model is of a certain type. The preset value is not limited and can be determined based on actual circumstances. For example, it can be, but is not limited to, 0.5, 0.7, 0.8, or 0.85.

[0035] As an embodiment, in this embodiment, images with a confidence level greater than a preset value are referred to as high-confidence images, and images with a confidence level less than or equal to the preset value are referred to as low-confidence images. If the above-mentioned preset value is set to 0.5, then the image classification model in the i-1th round of classification operation classifies image 1 as type B, and the corresponding first confidence level is 0.4. This first confidence level is less than the preset value 0.5, and image 1 is determined to be a low-confidence image. Assuming that i is 3, images with a first confidence level less than the preset value 0.5 output by the image classification model in the second round of classification operation can be obtained, that is, the set of low-confidence images output by the image classification model in the second round of classification operation.

[0036] Step S204: input the low-confidence image set into the image classification model in the i-th round of classification operation to obtain a high-confidence image set output in the i-th round of classification operation, wherein the high-confidence image set includes images whose second confidence determined in the i-th round of classification operation is greater than a preset value, and the second confidence is obtained by processing the corresponding images using the image classification model in the i-th round of classification operation.

[0037] Assuming i is 3, the low-confidence image set output by the image classification model in the second round of classification can be input into the image classification model in the third round of classification, resulting in a high-confidence image set output by the third round of classification. The images in the high-confidence image set are those for which the second confidence level output by the image classification model in the third round of classification is greater than a preset value. This preset value can be determined based on actual conditions, such as 0.5 or 0.7. Assuming that the third image classification model classifies image 2 as type A, and the corresponding second confidence level is 0.6, which is greater than the preset value of 0.5, image 2 is determined to be a high-confidence image, and type A is used as the type of image 2.

[0038] Through the above steps, the low-confidence image set output by the image classification model in the i-1th round of classification operation is input into the image classification model in the i-th round of classification operation to obtain the high-confidence image set output in the i-th round of classification operation. The image types of the high-confidence image set output by the image classification model in the i-th round of classification operation are used as the types of the high-confidence images. This achieves the purpose of classifying images using the image classification model and achieves the effect of automatically labeling image types. This can further solve the problem of low image type labeling efficiency caused by manual labeling of image types in the prior art.

[0039] Optionally, the execution entity of the above steps may be a terminal device or a server, etc., but is not limited thereto.

[0040] Optionally, after obtaining the high-confidence image set output in the i-th round of classification operation, the method further includes: obtaining the classification result of the image classification model in the i-th round of classification operation on the high-confidence image set; and determining the image type in the classification result as the type of the image in the high-confidence image set.

[0041] As an optional embodiment, each high-confidence image in the high-confidence image set output by the image classification model in each round of classification operation (including the i-1th round of classification operation and the i-th round of classification operation) corresponds to an image type. The classification result of the high-confidence image set by the image classification model in the i-th round of classification operation can be obtained, and the image type in the classification result is determined as the classification type of the image in the high-confidence image set. Assume that in the classification result of image 3 by the image classification model in the i-th round of classification operation, the classification type of image 3 is type C, and the corresponding second confidence level is 0.8, which is greater than the preset value 0.5, then image 3 is determined to be a high-confidence image, and the classification type of image 3 by the image classification model in the i-th round of classification operation is determined to be the type of image 3, that is, the type of image 3 is type C.

[0042] Optionally, before inputting the low-confidence image set into the image classification model in the i-th round of classification operation to obtain the high-confidence image set output in the i-th round of classification operation, the method further includes: acquiring the low-confidence image set and the high-confidence image set output in the i-1-th round of classification operation, wherein the high-confidence image set includes images whose first confidence level determined in the i-1-th round of classification operation is greater than the preset value; and using the low-confidence image set and the high-confidence image set output in the i-1-th round of classification operation to train the image classification model in the i-th round of classification operation.

[0043] As an optional implementation, the image classification model in the i-1th round of classification operation can be used to obtain an output low-confidence image set and a high-confidence image set. The images in the high-confidence image set are images whose first confidence determined in the i-1th round of classification operation is greater than a preset value, such as the preset value is 0.5. Then, all images with a first confidence greater than 0.5 obtained in the i-1th round of classification operation are classified into the high-confidence image set, and all images with a first confidence less than or equal to 0.5 obtained in the i-1th round of classification operation are classified into the low-confidence image set. The above-mentioned low-confidence image set and high-confidence image set can be used to train the image classification model in the i-1th round of classification operation.

[0044] As an optional implementation, the image classification model in the i-1th round of classification operation obtains the output low-confidence image set. The image classification model in the i-th round of classification operation is trained using the low-confidence image set and the labeled image samples.

[0045] Optionally, the use of the low-confidence image set and the high-confidence image set output in the i-1th round of classification operation to train the image classification model in the i-1th round of classification operation includes: using the low-confidence image set output in the i-1th round of classification operation to train the feature network to obtain the feature network in the i-1th round of classification operation; using the high-confidence image set output in the i-1th round of classification operation to train the classification network to obtain the classification network in the i-1th round of classification operation; and determining the feature network in the i-1th round of classification operation and the classification network in the i-1th round of classification operation as the image classification model in the i-1th round of classification operation.

[0046] As an optional implementation, the image classification model in each round of classification operation can be a classification network model based on contrastive learning. Assuming that the value of i is 2, the low-confidence image set and the high-confidence image set output in the first round of classification operation can be used to train the image classification model in the second round of classification operation. The low-confidence image set output in the first round of classification operation can be used to train the feature network to obtain the feature network used for image sample feature extraction in the second round of classification operation. The high-confidence image set output in the first round of classification operation can be used to train the classification network to obtain the classification network in the second round of classification operation. The feature network in the second round of classification operation and the classification network in the second round of classification operation are determined as the image classification model in the second round of classification operation.

[0047] Optionally, the using the high-confidence image set output in the i-1th round of classification operation to train the classification network to obtain the classification network in the i-th round of classification operation includes: using the high-confidence image set output in the i-1th round of classification operation and a labeled image set to train the classification network to obtain the classification network in the i-th round of classification operation, wherein the labeled image set includes images of known types.

[0048] As an optional implementation, assuming that the value of i is 2, the classification network can be trained using the set of high-confidence images output from the first round of classification operations to obtain the classification network used in the second round of classification operations. The classification network used in the second round of classification operations can be a classification network based on contrastive learning, and the images in the set of labeled images can be labeled image samples that have been classified. The classification network is trained using the set of high-confidence images output from the first round of classification operations and the set of labeled images to obtain the classification network used in the second round of classification operations.

[0049] Optionally, the feature network is trained using the set of low-confidence images output in the i-1th round of classification operation to obtain the feature network in the i-th round of classification operation, including: using the set of low-confidence images output in the i-1th round of classification operation to perform multiple iterative training on the feature network to obtain the feature network in the i-th round of classification operation, wherein the output of the feature network in the i-th round of classification operation satisfies a preset loss function, and the target parameter in the preset loss function increases according to a preset adjustment value as the number of iterations increases.

[0050] As an optional implementation, the low-confidence image set output by the i-1th round of classification operation can be used to train the feature network in the i-th round of classification operation. Assuming that i is 10, the low-confidence image set output by the 9th round of classification operation can be used to perform multiple iterative training on the feature network to obtain the feature network in the 10th round of classification operation. The number of iterative training times can be 10 times or 20 times, and the number of iterative training times can be determined based on actual conditions. If the output of the feature network in the 10th round of classification operation meets the preset convergence conditions and the feature network can be used to extract the features of the image, then the output of the feature network in the 10th round of classification operation can be a feature matrix, and the preset loss function can be a contrast loss function. A target parameter can be set in the contrast loss function, and the target parameter can increase according to a preset adjustment value as the number of iterations of the feature network increases.

[0051] As an embodiment, an image sample can be enhanced in different data enhancement methods to obtain multiple views, so that different views of the same sample are close enough in the feature space, and the distance between different samples is as far as possible. In the most primitive version, all samples except the image sample itself are other samples. A memory bank can be set up to store the features of historical samples. Figure 3 The figure shows an optional feature network training flow chart based on contrastive learning according to an embodiment of the present invention, wherein the image sample is subjected to different data enhancement methods to obtain two views, namely, the sample view and the image view. Figure 1 and sample view Figure 2 ,like Figure 3 The parameters of the feature network and prediction network of the upper and lower branches are the same. They respectively obtain the feature parameters Zi and Zj of the two sample views. The focus of the feature network is to learn more generalized features from a large number of samples. The goal of optimizing the prediction network parameters is more to achieve a relatively low loss and be closer to the task. When acquiring the feature network, the image samples have no annotated information. The focus is on learning more generalized features. The final output loss function is the contrast loss, which can be expressed as the following formula:

[0052]

[0053]

[0054] Among them, z in formula 1 i , z j represents the eigenvalues ​​of different views of the same image sample, N represents the total number of image samples, and the numerator exp(sim(z i , z j ) / τ) represents the similarity information of different views of the same sample. The denominator in Formula 1 Represents information about the similarity between different views. It aims to improve the cohesion of different views of the same sample while increasing its repulsion from other samples. Formula 2 represents the loss obtained by summing all combinations of samples.

[0055] The target parameter can be the parameter τ in formula 1. It can be seen that τ has a regulating effect. That is, the larger the target parameter τ is, the easier it is to aggregate the data, and the smaller τ is, the more dispersed the data is. Based on this, the present invention designs an automatically changing target parameter τ, which changes with the number of iterations, from small to large, as shown in the following formula 3:

[0056] τ=τ0+a×iter Formula 3

[0057] Where τ0 is the initial target parameter, iter is the number of iterations, and a is the preset adjustment value. In the early stages of the feature network iteration, more emphasis is placed on preserving its generalization performance. As the number of iterations increases, towards the end of the final iteration, more emphasis is placed on maintaining the similarity of its positive examples to obtain the final feature network.

[0058] Optionally, the classification network is trained using the high-confidence image set output in the i-1th round of classification operation to obtain the classification network in the i-th round of classification operation, including: using the high-confidence image set output in the i-1th round of classification operation to perform multiple iterative training on the classification network to obtain the classification network in the i-th round of classification operation, wherein the output value of the loss function between the estimated image type output by the classification network in the i-th round of classification operation and the known image type of the high-confidence image set satisfies a preset convergence condition.

[0059] As an optional implementation, the classification network in the i-th round of classification operation can be obtained by training the high-confidence image set output by the i-1th round of classification operation. Assuming that i is 10, the classification network can be iteratively trained multiple times using the high-confidence image set output by the 9th round of classification operation to obtain the classification network in the 10th round of classification operation. The number of iterative training times can be 10, 20, or 30 times, and the number of iterative training times can be determined based on actual conditions. The images in the high-confidence image set output by the 9th round of classification operation can be classified. When the output value of the loss function between the estimated image type output by the classification network in the 10th round of classification operation and the known image type in the high-confidence image set output by the 9th round of classification operation meets a preset convergence condition, the classification network in the 10th round of classification operation is determined, wherein the loss function can be a classification loss function.

[0060] Optionally, the feature network in the first round of classification operation is obtained by training with a labeled image set and an unlabeled image set, wherein the unlabeled image set includes images of unknown types and the labeled image set includes images of known types; the classification network in the first round of classification operation is obtained by training with the labeled image set; the feature network in the first round of classification operation and the classification network in the first round of classification operation are determined as the image classification model in the first round of classification operation.

[0061] As an optional implementation, the labeled image set includes images of known types, and the unlabeled image set includes images of unknown types. The feature network in the first round of classification operation can be obtained by training the labeled image set and the unlabeled image set. The training method is as follows: Figure 3 As shown in FIG, through the feature network training method based on contrastive learning, the feature network in the first round of classification operation is obtained by training the labeled image set and the unlabeled image set.

[0062] In addition, the above-mentioned labeled image set can be used to obtain the classification network in the first round of classification operation through supervised learning training. The classification network in the first round of classification operation can be a supervised classification network, such as Figure 4 The figure shows an optional supervised classification network training flow chart according to an embodiment of the present invention. The input image samples are sample enhanced, and the image feature parameters are extracted using the feature network. When the classification loss reaches the preset convergence condition, the classification network in the first round of classification operation is obtained. It should be noted that the parameters of the feature network do not need to be fixed during this training, and they are only used for initialization.

[0063] The feature network in the first round of classification operation and the classification network in the first round of classification operation obtained by the above training are used as the image classification model in the first round of classification operation of the iteration.

[0064] As an optional implementation, after the classification network in the first round of classification operation is obtained through the above training, all samples (unlabeled image sets) can be tested through the classification network in the first round of classification operation, and the data set can be divided into two parts according to the level of confidence. Assuming that the preset value is set to 0.6, the images with a confidence greater than 0.6 can be called the initial high-confidence image set, and the images with a confidence less than or equal to 0.6 can be called the initial low-confidence image set. The initial high-confidence image set and the labeled image set can be used to train the image classification model in the first round of classification operation.

[0065] As an optional implementation, the present invention provides an automated sample classification solution for extracting required samples from massive samples. This solution indirectly reveals how to train a model from scratch, but the present invention focuses on describing the overall process from the perspective of samples. Figure 5 The figure shows a flow chart of automatic classification samples based on contrastive learning according to an optional embodiment of the present invention. The specific steps are as follows:

[0066] S501, using all image samples (labeled and unlabeled), training based on contrastive learning to obtain the feature network in the first round of classification operation;

[0067] S502, using existing labeled samples, supervised training to obtain a classification network in the first round of classification operation;

[0068] S503: Classify all unlabeled samples using the classification network in the first round of classification. Select the classified samples with high confidence and assign them to the high confidence image set (H0), and the remaining samples to the low confidence image set (L0).

[0069] S504, obtaining a new contrastive learning-based classification network based on the feature network in the i-th round of classification operation through the high-confidence image set and the existing labeled samples;

[0070] S505, classifying the low-confidence image set through a new contrastive learning-based classification network to obtain a high-confidence image set (Hx) and a low-confidence image set (Lx);

[0071] S506, obtain a new feature network based on the low-confidence image set (Lx), return to step S504, and iterate, that is, train the high-confidence image set (Hx) and the existing labeled samples based on the new feature network to obtain the next new classification network based on contrastive learning.

[0072] Through the above steps, the low-confidence sample set is obtained by repeated iterative training, and it is iterated N times until the low-confidence samples are difficult to distinguish.

[0073] In the above step S504, based on the sample collection formed by the high confidence image set and the labeled samples, Figure 6 The training process in the training is carried out, Figure 6 This is a flowchart of a classification network training based on contrastive learning according to an optional embodiment of the present invention. Sample enhancement is performed on the input image sample, and the classification network is trained based on the feature network obtained in step S501. The loss function includes classification loss and contrast loss. It should be noted that there are two differences from the network model in step S502. First, the parameters of the feature network part are not updated. The feature network parameters obtained in step S501 are used for the first time, and the feature network parameters generated in step S506 are used during the iteration process. Second, step S504 adds contrast loss on the basis of the original classification loss.

[0074] Step S505 uses the network trained in step S504 to classify the low-confidence sample set after the last sample classification, and separates Hx and Lx according to the classification confidence.

[0075] Step S506: The dataset Lx generated in step S505 is Figure 3 The training is performed to obtain a new feature network based on contrastive learning. Then, steps S504 to S506 are repeated until the amount of high confidence image set data decreases significantly.

[0076] At this point, the required data is automatically extracted from a large amount of data, and there is basically no need for human intervention in the entire sample classification process. When the sample size is relatively large, this solution can be used to automatically filter out the required samples.

[0077] This invention provides a solution for automatic sample classification, automatically extracting the required samples from a large number of collected samples, avoiding significant manual effort. It utilizes contrastive learning, alternating between unsupervised and supervised approaches, to achieve a balance between network generalization and classification accuracy. The improved loss function for contrastive learning also achieves a balance between similarity and equilibrium during feature extraction.

[0078] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0079] This embodiment also provides a device for determining image type, which is used to implement the above-mentioned embodiments and preferred embodiments. Details already described will not be repeated here. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0080] Figure 7 is a structural block diagram of an image classification device according to an embodiment of the present invention, which is used to perform multiple rounds of classification operations on multiple images to be classified, such as Figure 7As shown, the device includes: an acquisition unit 72, used to acquire a low-confidence image set output in the i-1th round of classification operation, wherein the low-confidence image set includes images whose first confidence determined in the i-1th round of classification operation is less than or equal to a preset value, and the first confidence is obtained by processing the corresponding image using the image classification model in the i-1th round of classification operation, and i is greater than or equal to 2; an input unit 74, used to input the low-confidence image set into the image classification model in the i-th round of classification operation to obtain a high-confidence image set output in the i-th round of classification operation, wherein the high-confidence image set includes images whose second confidence determined in the i-th round of classification operation is greater than a preset value, and the second confidence is obtained by processing the corresponding image using the image classification model in the i-th round of classification operation.

[0081] Optionally, the above-mentioned device is also used to, after obtaining the high-confidence image set output in the i-th round of classification operation, obtain the classification result of the image classification model in the i-th round of classification operation on the high-confidence image set; and determine the image type in the classification result as the type of the image in the high-confidence image set.

[0082] Optionally, the above-mentioned device is also used to obtain the low-confidence image set and the high-confidence image set output in the i-1th round of classification operation before inputting the low-confidence image set into the image classification model in the i-th round of classification operation to obtain the high-confidence image set output in the i-1th round of classification operation, wherein the high-confidence image set includes images whose first confidence level determined in the i-1th round of classification operation is greater than the preset value; and use the low-confidence image set and the high-confidence image set output in the i-1th round of classification operation to train the image classification model in the i-1th round of classification operation.

[0083] Optionally, the above-mentioned device is also used to use the low-confidence image set output in the i-1th round of classification operation to train the feature network to obtain the feature network in the i-1th round of classification operation; use the high-confidence image set output in the i-1th round of classification operation to train the classification network to obtain the classification network in the i-th round of classification operation; and determine the feature network in the i-th round of classification operation and the classification network in the i-th round of classification operation as the image classification model in the i-th round of classification operation.

[0084] Optionally, the above-mentioned device is also used to train the classification network using the high-confidence image set output by the i-1th round of classification operation and the labeled image set to obtain the classification network in the i-th round of classification operation, wherein the labeled image set includes images of known types.

[0085] Optionally, the above-mentioned device is also used to use the low-confidence image set output by the i-1th round of classification operation to perform multiple iterative training on the feature network to obtain the feature network in the i-th round of classification operation, wherein the output of the feature network in the i-th round of classification operation satisfies a preset loss function, and the target parameter in the preset loss function increases according to a preset adjustment value as the number of iterations increases.

[0086] Optionally, the above-mentioned device is also used to use the high-confidence image set output by the i-1th round of classification operation to perform multiple iterative training on the classification network to obtain the classification network in the i-th round of classification operation, wherein the output value of the loss function between the estimated image type output by the classification network in the i-th round of classification operation and the known image type of the high-confidence image set satisfies a preset convergence condition.

[0087] Optionally, the above-mentioned device is also used to obtain the feature network in the first round of classification operation by training with a labeled image set and an unlabeled image set, wherein the unlabeled image set includes images of unknown types and the labeled image set includes images of known types; obtain the classification network in the first round of classification operation by training with the labeled image set; and determine the feature network in the first round of classification operation and the classification network in the first round of classification operation as the image classification model in the first round of classification operation.

[0088] It should be noted that the above modules can be implemented through software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0089] An embodiment of the present invention further provides a storage medium storing a computer program, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0090] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0091] S1, obtaining a low-confidence image set output by the image classification model in the i-1th round of classification operation, wherein the low-confidence image set includes images whose first confidence level determined in the i-1th round of classification operation is less than or equal to a preset value, the first confidence level being obtained by processing the corresponding images using the image classification model in the i-1th round of classification operation, and i is greater than or equal to 2;

[0092] S2. Input the low-confidence image set into the image classification model in the i-th round of classification operation to obtain a high-confidence image set output in the i-th round of classification operation, wherein the high-confidence image set includes images whose second confidence determined in the i-th round of classification operation is greater than a preset value, and the second confidence is obtained by processing the corresponding images using the image classification model in the i-th round of classification operation.

[0093] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0094] An embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0095] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0096] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0097] S1, obtaining a low-confidence image set output by the image classification model in the i-1th round of classification operation, wherein the low-confidence image set includes images whose first confidence level determined in the i-1th round of classification operation is less than or equal to a preset value, the first confidence level being obtained by processing the corresponding images using the image classification model in the i-1th round of classification operation, and i is greater than or equal to 2;

[0098] S2. Input the low-confidence image set into the image classification model in the i-th round of classification operation to obtain a high-confidence image set output in the i-th round of classification operation, wherein the high-confidence image set includes images whose second confidence determined in the i-th round of classification operation is greater than a preset value, and the second confidence is obtained by processing the corresponding images using the image classification model in the i-th round of classification operation.

[0099] Optionally, specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0100] Obviously, those skilled in the art will appreciate that the various modules or steps of the present invention described above can be implemented using a general-purpose computing device, centralized on a single computing device, or distributed across a network of multiple computing devices. Alternatively, they can be implemented using program code executable by a computing device, which can then be stored in a storage device and executed by the computing device. In some cases, the steps shown or described can be performed in a different order than that shown, or can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0101] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for classifying an image, characterized in that: include: Perform multiple rounds of classification operations on multiple images to be classified, where the i-th round of classification operations includes: Obtaining a set of low-confidence images output by the image classification model in the (i-1)th round of classification operation, wherein the low-confidence image set includes images having a first confidence level determined in the (i-1)th round of classification operation that is less than or equal to a preset value, the first confidence level being obtained by processing the corresponding images using the image classification model in the (i-1)th round of classification operation, and i being greater than or equal to 2; Obtaining a low-confidence image set and a high-confidence image set output in the (i-1)th round of classification operation, wherein the high-confidence image set includes images whose first confidence level determined in the (i-1)th round of classification operation is greater than the preset value; Using the low-confidence image set output in the (i-1)th round of classification operation to train the feature network, to obtain the feature network in the (i)th round of classification operation; Training the classification network using the high confidence image set output in the (i-1)th round of classification operation to obtain the classification network in the (i)th round of classification operation; Determining the feature network in the i-th round of classification operation and the classification network in the i-th round of classification operation as the image classification model in the i-th round of classification operation; The low-confidence image set is input into the image classification model in the i-th round of classification operation to obtain a high-confidence image set output in the i-th round of classification operation, wherein the high-confidence image set includes images whose second confidence determined in the i-th round of classification operation is greater than a preset value, and the second confidence is obtained by processing the corresponding images using the image classification model in the i-th round of classification operation.

2. The method according to claim 1, characterized in that After obtaining the high-confidence image set output in the i-th round of classification operation, the method further includes: Obtaining a classification result of the image classification model in the i-th round of classification operation on the high-confidence image set; The image type in the classification result is determined as the type of the image in the high-confidence image set.

3. The method according to claim 1, characterized in that The method of training a classification network using the high confidence image set outputted in the (i-1) round of classification operation to obtain the classification network in the (i) round of classification operation comprises: The classification network is trained using the high-confidence image set output by the (i-1)th round of classification operation and the labeled image set to obtain the classification network in the (i)th round of classification operation, wherein the labeled image set includes images of known types.

4. The method according to claim 1, wherein The feature network is trained using the low-confidence image set output in the i-1th round of classification operation to obtain the feature network in the i-th round of classification operation, including: The feature network is iteratively trained multiple times using the low-confidence image set output by the i-1th round of classification operation to obtain the feature network in the i-th round of classification operation, wherein the output of the feature network in the i-th round of classification operation satisfies a preset loss function, and the target parameter in the preset loss function increases according to a preset adjustment value as the number of iterations increases.

5. The method according to claim 3, characterized in that The classification network is trained using the high confidence image set output in the i-1th round of classification operation to obtain the classification network in the i-th round of classification operation, including: The classification network is iteratively trained multiple times using the high-confidence image set output by the i-1th round of classification operation to obtain the classification network in the i-th round of classification operation, wherein the output value of the loss function between the estimated image type output by the classification network in the i-th round of classification operation and the known image type of the high-confidence image set satisfies a preset convergence condition.

6. The method according to claim 1, characterized in that The method further comprises: A feature network in a first round of classification operation is obtained by training a set of labeled images and a set of unlabeled images, wherein the set of unlabeled images includes images of unknown types and the set of labeled images includes images of known types; Obtaining a classification network in a first round of classification operation by training the labeled image set; The feature network in the first round of classification operation and the classification network in the first round of classification operation are determined as the image classification model in the first round of classification operation.

7. An image classification device, characterized in that: Used to perform multiple rounds of classification operations on multiple images to be classified, including: an acquisition unit, configured to acquire a low-confidence image set output in an i-1th round of classification operation, wherein the low-confidence image set includes images having a first confidence level determined in the i-1th round of classification operation that is less than or equal to a preset value, the first confidence level being obtained by processing the corresponding images using the image classification model in the i-1th round of classification operation, and i being greater than or equal to 2; an input unit, configured to input the low-confidence image set into the image classification model in the i-th round of classification operation, to obtain a high-confidence image set output in the i-th round of classification operation, wherein the high-confidence image set includes images having a second confidence level greater than a preset value determined in the i-th round of classification operation, the second confidence level being obtained by processing the corresponding images using the image classification model in the i-th round of classification operation; The device is also used to obtain a low-confidence image set and a high-confidence image set output in the i-1th round of classification operation, wherein the high-confidence image set includes images whose first confidence is greater than the preset value determined in the i-1th round of classification operation; use the low-confidence image set output in the i-1th round of classification operation to train the feature network to obtain the feature network in the i-th round of classification operation; use the high-confidence image set output in the i-1th round of classification operation to train the classification network to obtain the classification network in the i-th round of classification operation; and determine the feature network in the i-th round of classification operation and the classification network in the i-th round of classification operation as the image classification model in the i-th round of classification operation.

8. A storage medium, characterized in that: The storage medium stores a computer program, wherein the program can be executed by a terminal device or a computer to execute the method described in any one of claims 1 to 6.

Images