Image processing model training method, device, electronic device and storage medium

Abstract

The present disclosure relates to a training method, device, electronic device, and storage medium for an image processing model, and relates to the field of image processing technology. The method comprises: extracting features from augmented image samples to be trained corresponding to image samples to be trained; performing loss processing based on the extracted feature vectors to obtain first classification losses, second classification losses, and comparison losses corresponding to the first and second augmented image samples to be trained corresponding to each image sample to be trained; determining the sample type of each image sample to be trained based on the first classification loss, second classification loss, and comparison loss; and iteratively training an initial image classification model based on the image samples to be trained in the order of sample type from simple to medium difficulty to difficult based on a curriculum learning algorithm to obtain a target image classification model. The present disclosure can solve the problem that the training effect of an image processing model is affected by noise samples.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a training method, device, electronic device, and storage medium for an image processing model. Background Art

[0002] During image training and processing, tagged images crawled from search engines using keywords or user-uploaded videos often contain a large amount of noise. Directly using these noise samples as training samples for image processing models can result in less than optimal training results. Improving the training performance of image processing models based on noise samples is a pressing issue. Summary of the Invention

[0003] The present disclosure provides a training method, device, electronic device, and storage medium for an image processing model, which can solve the problem that the training effect of the image processing model is affected by noise samples, thereby improving the accuracy of image processing model training. The technical solutions of the present disclosure are as follows:

[0004] According to a first aspect of an embodiment of the present disclosure, a method for training an image processing model is provided, comprising:

[0005] Performing two different forms of data augmentation on each of the image samples to be trained to obtain augmented image samples to be trained; the augmented image samples to be trained include a first augmented image sample to be trained and a second augmented image sample to be trained;

[0006] Extracting features from the augmented image samples to be trained, and obtaining a feature vector corresponding to each augmented image sample to be trained;

[0007] Calculating a loss value according to a first loss function and a feature vector corresponding to the augmented image sample to be trained, to obtain, for each image sample to be trained, a first classification loss corresponding to the first augmented image sample to be trained, a second classification loss corresponding to the second augmented image sample to be trained, and a comparison loss corresponding to the first augmented image sample to be trained and the second augmented image sample to be trained;

[0008] Determining the sample type of each to-be-trained image sample according to the first classification loss, the second classification loss, and the contrast loss;

[0009] Based on the curriculum learning algorithm, the image samples to be trained are sorted in order from simple, medium difficulty to difficult, and the initial image classification model is iteratively trained to obtain the target image classification model.

[0010] In some embodiments of the present disclosure, the step of calculating the loss value according to the first loss function and the feature vector corresponding to the augmented image sample to be trained to obtain the first classification loss corresponding to the first augmented image sample to be trained, the second classification loss corresponding to the second augmented image sample to be trained, and the comparison loss corresponding to the first augmented image sample to be trained and the second augmented image sample to be trained includes:

[0011] Calculating a classification loss value according to the first loss function and the feature vector corresponding to the augmented image sample to be trained, to obtain a first classification loss corresponding to the first augmented image sample to be trained and a second classification loss corresponding to the second augmented image sample to be trained for each image sample to be trained;

[0012] Calculate the distance between the first eigenvector and the second eigenvector corresponding to each image sample to be trained, and obtain the comparison loss corresponding to the first augmented image sample to be trained and the second augmented image sample to be trained for each image sample to be trained, where the first eigenvector is the eigenvector corresponding to the first augmented image sample to be trained, and the second eigenvector is the eigenvector corresponding to the second augmented image sample to be trained.

[0013] In some embodiments of the present disclosure, determining the sample type of each to-be-trained image sample according to the first classification loss, the second classification loss, and the contrast loss includes:

[0014] In response to the contrast loss being less than a first contrast loss threshold, and the first classification loss and the second classification loss being both less than the first classification loss threshold, determining that the to-be-trained sample is a first simple sample, where the first simple sample represents an image sample that is correctly labeled and has a probability of being recognized greater than a first recognition threshold;

[0015] In response to the contrast loss being less than a first contrast loss threshold, and the first classification loss and the second classification loss being both greater than a second classification loss threshold, determining that the to-be-trained sample is a second simple sample, where the second simple sample represents an image sample that is incorrectly labeled but has a probability of being recognized greater than the first recognition threshold;

[0016] In response to the contrast loss being greater than a second contrast loss threshold, and the first classification loss and the second classification loss having a classification loss greater than the second classification loss threshold and a classification loss less than the first classification loss threshold, determining that the to-be-trained sample is a medium-difficulty sample, the second classification loss threshold being greater than the first classification loss, and the medium-difficulty sample representing an image sample having a probability of being recognized greater than the second recognition threshold and less than the first recognition threshold;

[0017] In response to the contrast loss being greater than a second contrast loss threshold, and the first classification loss and the second classification loss being both greater than the second classification loss threshold, determining that the to-be-trained sample is a difficult sample, where the difficult sample represents an image sample whose probability of being recognized is less than a second recognition threshold;

[0018] The difficulty of identifying the first simple sample, the second simple sample, the medium-difficulty sample, and the difficult sample increases in sequence.

[0019] In some embodiments of the present disclosure, the step of iteratively training the initial image classification model based on the curriculum learning algorithm according to the order of sample types from simple to medium difficulty to difficult to obtain the target image classification model includes:

[0020] According to a first predetermined step length, the first simple sample is input into an initial image classification model for training to obtain a first image classification model;

[0021] According to a second predetermined step length, the first simple sample and the second simple sample are input into a first image classification model for training to obtain a second image classification model;

[0022] According to a third predetermined step length, the first simple sample, the second simple sample, and the moderately difficult sample are input into the second image classification model for training to obtain a third image classification model;

[0023] According to a fourth predetermined step length, the first simple sample, the second simple sample, the medium-difficulty sample, and the difficult sample are input into a third image classification model for training to obtain a target image classification model;

[0024] Wherein, the first predetermined step length, the second predetermined step length, the third predetermined step length, and the fourth predetermined step length decrease in sequence;

[0025] The sample weights of the first simple sample, the second simple sample, the medium-difficulty sample and the difficult sample are the largest during the first training and gradually decrease during the subsequent training process. The sum of the sample weights of each training of the image processing model is 1.

[0026] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing method, the method further comprising:

[0027] The image to be processed is input into the target image classification model trained by any image processing model training method of the first aspect to perform image processing to obtain the category of the image to be processed.

[0028] According to a third aspect of an embodiment of the present disclosure, there is provided a training device for an image processing model, comprising:

[0029] a data processing unit configured to perform two different forms of data augmentation on each to-be-trained image sample in the to-be-trained image samples to obtain augmented to-be-trained image samples; the augmented to-be-trained image samples comprising a first augmented to-be-trained image sample and a second augmented to-be-trained image sample;

[0030] a feature extraction unit configured to perform feature extraction on the augmented image samples to be trained, and obtain a feature vector corresponding to each augmented image sample to be trained;

[0031] a loss acquisition unit configured to calculate a loss value based on a first loss function and a feature vector corresponding to the augmented image sample to be trained, to obtain, for each image sample to be trained, a first classification loss corresponding to the first augmented image sample to be trained, a second classification loss corresponding to the second augmented image sample to be trained, and a comparison loss corresponding to the first augmented image sample to be trained and the second augmented image sample to be trained;

[0032] a sample type determination unit, configured to determine the sample type of each to-be-trained image sample according to the first classification loss, the second classification loss, and the contrast loss;

[0033] The model training unit is configured to iteratively train the initial image classification model based on the curriculum learning algorithm according to the order of sample types from simple, medium difficulty to difficult to obtain the target image classification model.

[0034] According to a fourth aspect of an embodiment of the present disclosure, there is provided an image processing apparatus, including:

[0035] The image processing unit is configured to input the image to be processed into the target image classification model trained by the training method of the image processing model described in the first aspect to perform image processing to obtain the category of the image to be processed.

[0036] According to a fifth aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

[0037] processor;

[0038] a memory for storing instructions executable by the processor;

[0039] The processor is configured to execute the instructions to implement the image processing model training method as described in the first aspect or the image processing method as described in the second aspect.

[0040] According to the sixth aspect of an embodiment of the present disclosure, a computer storage medium is provided. When the instructions in the computer storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the training method of the image processing model as described in the first aspect above or the image processing method as described in the second aspect.

[0041] According to the seventh aspect of the embodiments of the present disclosure, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the training method of the image processing model as described in the first aspect above or the image processing method as described in the second aspect.

[0042] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects: In the embodiments of the present disclosure, when image samples to be trained are received, data augmentation in different forms is performed twice on each image sample to be trained in the image samples to obtain augmented image samples to be trained; feature extraction is performed on the augmented image samples to be trained to obtain a feature vector corresponding to each augmented image sample to be trained, and a loss value is calculated based on a first loss function and the feature vector corresponding to the augmented image sample to be trained to obtain a first classification loss of the first augmented image sample to be trained, a second classification loss corresponding to the second augmented image sample to be trained, and a comparison loss corresponding to the first augmented image sample to be trained and the second augmented image sample to be trained; and based on the first classification loss, the second classification loss and the comparison loss, the sample type of each image sample to be trained is determined, and based on the curriculum learning algorithm, the image samples to be trained are iteratively trained on the initial image classification model in the order of the sample types from simple, medium difficulty to difficult to obtain a target image classification model. The training process of the entire image processing model is no longer limited to directly inputting the acquired image samples to be trained into the image classification model for training without distinction. Instead, based on the strongly supervised contrast loss and the self-supervised classification loss, the training image samples are classified into the types of training difficulty samples, and simple samples and difficult samples are distinguished. Based on the curriculum learning algorithm, the image samples to be trained are ordered from simple, medium difficulty to difficult in the order of their sample types, and the initial image classification model is iteratively trained. This solves the problem that the training effect of the image processing model is affected by noise samples, thereby improving the accuracy of image processing model training.

[0043] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0045] Figure 1 is a flowchart of a method for training an image processing model according to an exemplary embodiment;

[0046] Figure 2 is a schematic diagram of data augmentation processing in a training method of an image processing model according to an exemplary embodiment;

[0047] Figure 3 is a flowchart of a method for training an image processing model according to an exemplary embodiment;

[0048] Figure 4 is a flowchart of a method for training an image processing model according to an exemplary embodiment;

[0049] Figure 5 is a flowchart of course learning in a training method for an image processing model according to an exemplary embodiment;

[0050] Figure 6 is a flowchart of an image processing method according to an exemplary embodiment;

[0051] Figure 7 is a block diagram of a training device for an image processing model according to an exemplary embodiment;

[0052] Figure 8 is a block diagram of a training device for an image processing model according to an exemplary embodiment;

[0053] Figure 9 is a block diagram of an image processing apparatus according to an exemplary embodiment;

[0054] Figure 10 is a block diagram of an electronic device according to an exemplary embodiment;

[0055] Figure 11 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0056] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0057] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0058] During image training and processing, tagged images crawled from search engines using keywords or user-uploaded videos often contain a large amount of noise. Directly using these noise samples as training samples for image processing models can result in less than optimal training results. Improving the training performance of image processing models based on noise samples is a pressing issue.

[0059] In order to solve the above technical problems, the present invention provides a training method, device, electronic device and storage medium for an image processing model, which can solve the problem that the image processing model is affected by noise samples and the training effect is affected, thereby improving the accuracy of image processing model training. Figure 1 As mentioned above, the embodiments of the present disclosure provide a method for training an image processing model, comprising:

[0060] Step 101: perform two different forms of data augmentation on each of the training image samples to obtain augmented training image samples; the augmented training image samples include a first augmented training image sample and a second augmented training image sample.

[0061] When training an image processing model, image samples to be trained are first obtained. In the embodiments of the present disclosure, the image samples to be trained can be pictures to be trained, and the embodiments of the present disclosure do not limit the number of such images. After obtaining the image samples to be trained, each of the image samples to be trained is subjected to two different forms of data augmentation to obtain augmented image samples to be trained. The augmented image samples to be trained include a first augmented image sample to be trained and a second augmented image sample to be trained. The first augmented image sample to be trained is obtained by one form of data augmentation, and the second augmented image sample to be trained is obtained by another form of data augmentation.

[0062] It should be noted here that, when performing two different forms of data augmentation on each to-be-trained image sample in the embodiments of the present disclosure, the augmented to-be-trained image sample can be obtained by randomly cropping the to-be-trained image sample, or the augmented to-be-trained image sample can be obtained by transforming the image brightness, color difference, contrast, grayish white, etc. As long as the two augmented to-be-trained image samples obtained are different samples, that is, the two augmented images obtained are different images, the specific method adopted is not limited by the embodiments of the present disclosure.

[0063] Step 102: extract features from the augmented image samples to be trained, and obtain a feature vector corresponding to each augmented image sample to be trained.

[0064] In the embodiments of the present disclosure, when extracting features from the augmented image samples to be trained, the methods may be, but are not limited to, inputting the augmented image samples to be trained into a deep neural network for feature extraction. Other feature extraction methods may also be used, and the specific embodiments of the present disclosure are not limited to this.

[0065] In the embodiments of the present disclosure, when a deep neural network is used for feature extraction, feature extraction can be performed using, but is not limited to, resnet50, inception-v3, etc. Specifically, in the embodiments, feature extraction corresponding to 2048 dimensions or 1024 dimensions can be performed but is not limited to.

[0066] Step 103: Calculate the loss value according to the first loss function and the feature vector corresponding to the augmented image sample to be trained, and obtain the first classification loss corresponding to the first augmented image sample to be trained, the second classification loss corresponding to the second augmented image sample to be trained, and the comparison loss corresponding to the first augmented image sample to be trained and the second augmented image sample to be trained for each image sample to be trained.

[0067] In the embodiments of the present disclosure, when calculating the loss value based on the loss function and the feature vector, the classification loss and contrast loss used to classify the training difficulty are mainly calculated. When calculating the classification loss, it is achieved through strongly supervised image classification, and when calculating the contrast loss, it is achieved based on self-supervised feature vector comparison. The specific embodiments of the present disclosure are not limited to this.

[0068] Step 104: Determine the sample type of each to-be-trained image sample according to the first classification loss, the second classification loss, and the contrast loss.

[0069] In the embodiment of the present disclosure, the sample types include: first simple sample, second simple sample, medium difficulty sample and difficult sample. Specifically:

[0070] The first simple sample represents an image sample that is correctly labeled and has a probability of being recognized that is greater than a first recognition threshold. The second simple sample represents an image sample that is incorrectly labeled but has a probability of being recognized that is greater than the first recognition threshold. The moderately difficult sample represents an image sample that has a probability of being recognized that is greater than the second recognition threshold and less than the first recognition threshold. The difficult sample represents an image sample that has a probability of being recognized that is less than the second recognition threshold. It should be noted here in the embodiments of the present disclosure that the first recognition threshold and the second recognition threshold are empirical values ​​and can be dynamically adjusted according to actual conditions, such as the first recognition threshold is 90 and the second recognition threshold is 50; the specific embodiments of the present disclosure do not limit this.

[0071] Step 105: Based on the curriculum learning algorithm, the image samples to be trained are iteratively trained on the initial image classification model in the order of sample types from simple, medium difficulty to difficult to obtain the target image classification model.

[0072] In the embodiments of the present disclosure, based on a curriculum learning algorithm, when the image samples to be trained are iteratively trained on the initial image classification model in the order of sample types from simple, medium difficulty to difficult to obtain a target image classification model, the method can be adopted but not limited to curriculum learning, and the specific embodiments of the present disclosure are not limited to this.

[0073] Curriculum learning is a strategy for training neural networks that mimics the human learning process. It advocates using the model to first learn simple samples or tasks, then learn complex samples or tasks, thereby enabling the model to achieve stronger generalization and convergence speed. If the model is trained using curriculum learning, the data samples need to be labeled with the training difficulty level during the curriculum learning-based model training process. In the embodiments of the present disclosure, the sample type of training difficulty of each image sample to be trained is determined based on the first classification loss, the second classification loss, and the contrast loss, thereby labeling the training difficulty level of the image samples to be trained.

[0074] In order for the image processing model to obtain stronger generalization ability and convergence speed, the course learning can train the image processing model by training the image samples to be trained in the order of training difficulty from simple to difficult. The specific embodiments of the present disclosure are not limited to this.

[0075] In an embodiment of the present disclosure, when image samples to be trained are received, data augmentation in different forms is performed twice on each image sample to be trained in the image samples to obtain augmented image samples to be trained; feature extraction is performed on the augmented image samples to be trained to obtain a feature vector corresponding to each augmented image sample to be trained, and a loss value is calculated based on a first loss function and the feature vector corresponding to the augmented image sample to be trained to obtain a first classification loss of the first augmented image sample to be trained, a second classification loss corresponding to the second augmented image sample to be trained, and a comparison loss corresponding to the first augmented image sample to be trained and the second augmented image sample to be trained; and based on the first classification loss, the second classification loss and the comparison loss, the sample type of each image sample to be trained is determined, and based on the curriculum learning algorithm, the image samples to be trained are iteratively trained on an initial image classification model in the order of the sample types to which they belong from simple, medium difficulty to difficult, to obtain a target image classification model. The training process of the entire image processing model is no longer limited to directly inputting the acquired image samples to be trained into the image classification model for training without distinction. Instead, based on the strongly supervised contrast loss and the self-supervised classification loss, the training image samples are classified into the types of training difficulty samples, and simple samples and difficult samples are distinguished. Based on the curriculum learning algorithm, the image samples to be trained are ordered from simple, medium difficulty to difficult in the order of their sample types, and the initial image classification model is iteratively trained. This solves the problem that the training effect of the image processing model is affected by noise samples, thereby improving the accuracy of image processing model training.

[0076] In some embodiments of the present disclosure, Figure 2 As shown in FIG, when the number of image samples to be trained is N, that is, when there are N image samples to be trained, two different forms of data augmentation are performed on each of the N image samples to be trained using the above data augmentation method. For example, Figure 2 Two image samples 1 and 2 are randomly cropped from the image. After data augmentation, N*2 image samples are obtained. Based on this embodiment, when performing feature extraction on the augmented image samples to be trained, the N*2 image samples can be input into a deep neural network to extract feature vectors.

[0077] In some embodiments of the present disclosure, when the loss value is calculated according to the first loss function and the feature vector, the first classification loss corresponding to the first augmented image sample to be trained, the second classification loss corresponding to the second augmented image sample to be trained, and the comparison loss corresponding to the first augmented image sample to be trained and the second augmented image sample to be trained are obtained. Figure 3 As shown, this can be achieved by, but not limited to, the following method, which includes:

[0078] 1031. Calculate a classification loss value based on the first loss function and the feature vector corresponding to the augmented image sample to be trained, and obtain a first classification loss corresponding to the first augmented image sample to be trained and a second classification loss corresponding to the second augmented image sample to be trained for each image sample to be trained.

[0079] It should be noted here that when calculating the classification loss, it is generally calculated by processing the loss function through strong supervised learning. When specifically implemented, it can be implemented by but not limited to the following method: the feature vector of the augmented image sample to be trained with a dimension of D is input into the multi-layer image classification network model, and the classification loss value is calculated to obtain the classification loss.

[0080] Based on the above Figure 2 For example, the feature vector is input into the image classification network model, and the classification loss value of the feature vector corresponding to the augmented image sample to be trained is calculated according to the first loss function in the image classification network model to obtain the first classification loss corresponding to the first augmented image sample to be trained and the second classification loss corresponding to the second augmented image sample to be trained for each image sample to be trained, including: inputting the feature vector with the above-mentioned N*2 image dimension D into the multi-layer image classification network model (FC,) to realize the calculation of the classification loss value, and obtain C-L1-classification loss, assuming it is C-Loss i1 and C-Loss i2 , the C-Loss i1 and C-Loss i2 is the classification loss of the two views of the i-th sample in the training image sample.

[0081] 1032. Calculate the distance between the first eigenvector and the second eigenvector corresponding to each image sample to be trained, and obtain the comparison loss corresponding to the first augmented image sample to be trained and the second augmented image sample to be trained for each image sample to be trained, where the first eigenvector is the eigenvector corresponding to the first augmented image sample to be trained, and the second eigenvector is the eigenvector corresponding to the second augmented image sample to be trained.

[0082] It should be noted here that when calculating the contrast loss, it can be obtained by calculating the distance between the first eigenvector and the second eigenvector, and the specific embodiments of the present disclosure are not limited to this.

[0083] Based on the above Figure 2For example, calculating the distance between the first eigenvector and the second eigenvector corresponding to each image sample to be trained, and obtaining the comparison loss corresponding to the first augmented image sample to be trained and the second augmented image sample to be trained for each image sample to be trained includes: simultaneously calculating the eigenvector distance between the two eigenvectors of the image dimension D of the i-th image sample, that is,

[0084] S-Loss=||D i1 -D i2 ||)

[0085] Among them, S-Loss is Figure 2 The L2 contrast loss in , D(i1) represents the feature vector generated by the first augmented image sample of the i-th image sample, D(i2) represents the feature vector generated by the second augmented image sample of the i-th image sample, and the |||| symbol represents the distance metric between the two vectors, which is generally the cosine distance.

[0086] In some embodiments of the present disclosure, as described above, using the curriculum learning training model, it is necessary to classify the training sample data by the degree of training difficulty. When classifying the training image samples by the degree of training difficulty, the sample type of each image sample to be trained can be determined based on the first classification loss, the second classification loss, and the contrast loss. When determining the sample type of each image sample to be trained based on the first classification loss, the second classification loss, and the contrast loss, the following methods can be used but are not limited to achieve this: Figure 4 As shown, the method includes:

[0087] 1. First simple sample X0: In response to the contrast loss being less than the first contrast loss threshold, and the first classification loss and the second classification loss being both less than the first classification loss threshold, the sample to be trained is determined to be a first simple sample, where the first simple sample represents an image sample that is correctly labeled and has a probability of being recognized greater than a first recognition threshold.

[0088] It should be noted here that when the contrast loss corresponding to the two augmented image samples to be trained of a to-be-trained image sample is less than the first contrast threshold, and the first classification loss and the second classification loss corresponding to the two augmented image samples to be trained are both less than the first classification loss threshold, it means that this type of sample is easy to be recognized no matter how the view changes, so the to-be-trained image sample is determined to be the first simple sample X0.

[0089] The first contrast loss threshold and the first classification loss threshold in the embodiment of the present disclosure can be empirical values. In the specific implementation of the embodiment of the present disclosure, the values ​​are set according to experience. For example, the first contrast loss threshold is 0.05 and the first classification loss threshold is 0.1. When the contrast loss corresponding to the two augmented image samples to be trained of a to-be-trained image sample is less than 0.05, and the first classification loss and the second classification loss corresponding to the two augmented image samples to be trained are both less than 0.1, the to-be-trained sample is determined to be the first simple sample X0. That is, S-Loss is less than 0.05, and C-Loss is less than 0.05. i1 and C-Loss i2 If both are less than 0.1, the i-th image sample to be trained is determined to be the first simple sample X0.

[0090] 2. Second simple sample X1: In response to the contrast loss being less than the first contrast loss threshold, and the first classification loss and the second classification loss being both greater than the second classification loss threshold, the sample to be trained is determined to be a second simple sample, and the second simple sample represents an image sample that is incorrectly labeled but has a probability of being recognized greater than the first recognition threshold.

[0091] It should be noted here that when the contrast loss corresponding to the two augmented image samples to be trained of a sample to be trained is less than the first contrast threshold, and the first classification loss and the second classification loss corresponding to the two augmented image samples to be trained are both greater than the second classification loss threshold, it means that no matter how the image of this type of sample changes, the images are very similar, but are easily misidentified, so the sample to be trained is determined to be the second simple sample X1.

[0092] The second classification loss threshold in the embodiment of the present disclosure can be an empirical value. In the specific implementation of the embodiment of the present disclosure, the value is set according to experience. For example, the second classification loss threshold is 0.7. When the contrast loss of two augmented training image samples of a training image sample is less than 0.05, and the first classification loss and the second classification loss corresponding to the two augmented training image samples are both greater than 0.7, the training sample is determined to be the second simple sample X1. That is, S-Loss is less than 0.05, and C-Loss is less than 0.05. i1 and C-Loss i2 If both are greater than 0.7, the i-th image sample to be trained is determined to be the second simple sample X1.

[0093] 3. Moderately difficult sample X2: In response to the contrast loss being greater than the second contrast loss threshold, and the first classification loss and the second classification loss having a classification loss greater than the second classification loss threshold, and also having a classification loss less than the first classification loss threshold, the sample to be trained is determined to be a moderately difficult sample, the second classification loss threshold is greater than the first classification loss, and the moderately difficult sample represents an image sample whose probability of being recognized is greater than the second recognition threshold and less than the first recognition threshold.

[0094] It should be noted here that when the contrast loss corresponding to the two augmented image samples to be trained of an image sample to be trained is greater than the second contrast threshold, it means that the scale of change of the image sample is large; and there is a classification loss greater than the second classification loss threshold and a classification loss less than the first classification loss threshold in the first classification loss and the second classification loss corresponding to the two augmented image samples to be trained, it can be judged that some images in the image sample to be trained can be recognized, for example, areas related to the category, and some images in the image sample to be trained cannot be recognized, so the sample to be trained is determined to be a labeling medium-difficulty sample X2.

[0095] The second contrast loss threshold in the embodiment of the present disclosure can be an empirical value. In the specific implementation of the embodiment of the present disclosure, the value is set according to experience. For example, the second contrast loss threshold is 0.3. When the contrast loss corresponding to the two augmented image samples to be trained is greater than 0.3, and the first classification loss and the second classification loss corresponding to the two augmented image samples to be trained have a classification loss greater than 0.7 and a classification loss less than 0.1, the training sample is determined to be a medium-difficulty sample X2. That is, S-Loss is greater than 0.3, and C-Loss is less than 0.1. i1 and C-Loss i2 If some are greater than 0.7 and some are less than 0.1, the i-th image sample to be trained is determined to be a moderately difficult sample X2 to be labeled.

[0096] It should be noted that the embodiments of the present disclosure include medium-difficulty samples that are correctly labeled and medium-difficulty samples that are incorrectly labeled. Since it is difficult to distinguish whether the recognition difficulty of medium-difficulty samples is caused by incorrect labeling or other reasons, it is better to distinguish between correct and incorrectly labeled simple samples and unify them into medium-difficulty samples.

[0097] 4. Difficult sample X3: In response to the contrast loss being greater than the second contrast loss threshold, and the first classification loss and the second classification loss being greater than the second classification loss threshold, the sample to be trained is determined to be a difficult sample, and the difficult sample represents an image sample whose probability of being recognized is less than the second recognition threshold.

[0098] It should be noted here that when the contrast loss corresponding to the two augmented image samples to be trained of a sample to be trained is greater than the second contrast threshold, it indicates that the scale of change of the image sample is large; and the first classification loss and the second classification loss corresponding to the two augmented image samples to be trained are both greater than the second classification loss threshold, indicating that the sample is difficult to be recognized by the model, so the sample to be trained is determined to be a difficult-to-label sample X3.

[0099] In the specific implementation of the embodiment of the present disclosure, when the contrast loss of two augmented training image samples of a training image sample is greater than 0.3, and the first classification loss and the second classification loss corresponding to the two augmented training image samples are both greater than 0.7, the training sample is determined to be a difficult-to-label sample X3. That is, S-Loss is greater than 0.3, and C-Loss is greater than 0.7. i1 and C-Loss i2 If both are greater than 0.7, the i-th image sample to be trained is determined to be a difficult-to-label sample X3.

[0100] It should be noted that the embodiments of the present disclosure include medium-difficulty samples that are correctly labeled and medium-difficulty samples that are incorrectly labeled. Since it is difficult to distinguish whether the recognition difficulty of medium-difficulty samples is caused by incorrect labeling or other reasons, it is better to distinguish between correct and incorrectly labeled simple samples and unify them into medium-difficulty samples.

[0101] Based on the above description, the loss of a single sample Xi = (C-Loss i1 +C-Loss i2 ) / / 2.0+S-Loss, so the loss of the total sample Loss=a0*X0+a1*X1+a2*X2+a3*X3 (where a0+a1+a2+a3=1).

[0102] In some embodiments of the present disclosure, based on the curriculum learning algorithm, when the image samples to be trained are iteratively trained on the initial image classification model in the order of sample types from simple to medium difficulty to difficult to obtain the target image classification model, based on the above description, the following method can be used but is not limited to implement it, including:

[0103] According to a first predetermined step length, the first simple sample is input into an initial image classification model for training to obtain a first image classification model;

[0104] Inputting the first simple sample and the second simple sample into a first image classification model for training according to a second predetermined step length to obtain a second image classification model;

[0105] According to a third predetermined step length, the first simple sample, the second simple sample, and the moderately difficult sample are input into a second image classification model for training to obtain a third image classification model;

[0106] According to a fourth predetermined step length, the first simple sample, the second simple sample, the medium-difficulty sample, and the difficult sample are input into a third image classification model for training to obtain a target image classification model;

[0107] Wherein, the first predetermined step length, the second predetermined step length, the third predetermined step length, and the fourth predetermined step length decrease in sequence;

[0108] The sample weights of the first simple sample, the second simple sample, the medium-difficulty sample and the difficult sample are the largest during the first training and gradually decrease during the subsequent training process. The sum of the sample weights of each training of the image processing model is 1.

[0109] Among them, the specific steps based on course learning can refer to the following content, such as Figure 5 Shown, including:

[0110] S0: First, focus on learning the first simple sample so that the image classification model has simple learning capabilities.

[0111] In the first simple sample training phase, the training step size can be, but is not limited to, 1 / 3 of the step size epochs in the core area. This allows focus on the 1 / 3 epochs in the core area of ​​the first simple sample. The sample weights can be: a0 = 1, a1 = 0, a2 = 0, a3 = 0. In this training phase, the first simple sample is input into the initial image classification model for training according to the first predetermined step size to obtain the first image classification model. Samples of other classes are not trained in this training phase, so the coefficient a0 of the first simple sample is set to 1.

[0112] S1: Then focus on learning the second simple sample, while taking into account the first simple sample to avoid forgetting the learning ability, so that the image classification model has better learning ability.

[0113] In the second simple sample training stage, the training step size can be but not limited to the core area 1 / 4 epochs, so that the core area 1 / 4 epochs of the second simple sample can be focused on, and the sample weights can be: a0=0.2, a1=0.8, a2=0, a3=0.

[0114] It should be noted that in the embodiments of the present disclosure, from the training iteration of the first simple sample to the training of the first simple sample and the second simple sample, because the training of the image processing model involves the training of the second simple sample, which is more difficult than the first simple sample, the training step size is reduced from the original 1 / 3 epochs to 1 / 4 epochs. This ensures that an appropriate number of samples are selected for training. The reduction in step size can be 1 / 4 epochs in the embodiments of the present disclosure, or other values, as long as there is an overall trend of reduction. The specific setting can be based on actual conditions and is not limited in the embodiments of the present disclosure.

[0115] As mentioned above, this iteration involves both the first and second simple samples, but the training is primarily focused on the second simple sample. Therefore, when setting the weight of the second simple sample, the weight of the second simple sample will be increased. The weight a1 of the second simple sample is set to 0.8, the weight of the first simple sample is reduced, and a0 is set to 0.2. The sum of the weight a0 of the first simple sample and the weight a1 of the second simple sample in the training of this image processing model is 1.

[0116] S2: Then learn the medium-difficulty samples and try to get such samples.

[0117] Among them, in the medium-difficulty sample training stage, the training step size can be but not limited to the core area 1 / 4 epochs, so that the core area 1 / 4 epochs of the medium-difficulty sample can be focused on, and the sample weights can be: a0=0.2, a1=0.3, a2=0.5, a3=0.

[0118] It should be noted here in the embodiments of the present disclosure that the iterative training of this round of image processing models involves a first simple sample, a second simple sample, and a moderately difficult sample. Since this round of iterative training of the image processing model involves moderately difficult samples, as the difficulty of the samples increases, a moderate number of moderately difficult samples, rather than a large number, will be input into the image classification model for training. Based on this theory, the step size of this round of training can be maintained at 1 / 4 epochs or reduced, as long as there is an overall trend of reduction, for example, it can also be 1 / 5 epochs. The specific setting can be based on actual conditions, and the embodiments of the present disclosure are not limited to this.

[0119] As mentioned above, this iterative training involves the first simple sample, the second simple sample and the medium difficulty sample, but it is mainly trained for the medium difficulty sample, so when setting the sample weight, the weight of the medium difficulty sample is set to the weight with the largest sample weight among all samples. The medium difficulty sample weight a2 is set to 0.5, the second simple sample weight is reduced relative to the weight of the second iteration, a1 is reduced from the original 0.8 to 0.3, and the first simple sample weight a0 is set to 0.2 and remains unchanged. You can also set a1 to 0.35 and a0 to 0.15, as long as there is a decreasing trend as a whole, and the embodiments of the present disclosure are not limited to this. In the training of this image processing model, the sum of the first simple sample weight a0, the second simple sample weight a1 and the medium difficulty sample weight a2 can be 1.

[0120] S3: Finally, we learn difficult samples, hoping to generalize to more difficult samples with the help of the capabilities of the previous types of samples.

[0121] In the difficult sample training phase, the training step size can be, but is not limited to, 1 / 6 epochs in the core area, so that attention can be paid to the 1 / 6 epochs in the core area of ​​the difficult sample. The sample weights can be: a0 = 0.15, a1 = 0.25, a2 = 0.3, a3 = 0.3.

[0122] It should be noted that the iterative training of the image processing model in this round involves a first simple sample, a second simple sample, a moderately difficult sample, and a difficult sample. Because this round of iterative training involves difficult samples, as the difficulty of the samples increases, relatively fewer samples are input into the image classification model for training. Based on this theory, the step size of this round of training can be maintained at 1 / 6 epochs. This step size can be set based on actual conditions and is not limited in this embodiment of the present disclosure.

[0123] As mentioned above, this iteration involves the first simple sample, the second simple sample, the medium-difficulty sample, and the difficult sample, but it is mainly trained on the difficult samples. However, the training of difficult samples is more difficult. In order to minimize the impact of difficult samples on the image classification model, when setting the sample weights, the weight of the difficult sample is set to the largest weight among all samples but not too large. For example, the difficult sample weight a3 is set to 0.3, the medium-difficulty sample weight a2 is set to 0.3, the second simple sample weight a1 is set to 0.25, and the first simple sample weight a0 is set to 0.15. For the training of this image processing model, the sum of the first simple sample weight a0, the second simple sample weight a1, the medium-difficulty sample weight a2, and the difficult sample weight a3 can be 1.

[0124] It should be noted that during training, the sample weights and training epochs can be fine-tuned, but they must meet the overall trend of change, that is, avoid focusing too much on difficult samples (i.e., noise samples), and the number of epochs gradually decreases; the sample weight a0 gradually decreases, a1 gradually decreases from S1, and ai gradually decreases from Si (i = 1, 2, 3). During specific implementation, the reduction can be based on actual conditions, and this embodiment of the present invention is not limited to this.

[0125] Training of the image processing model described above can be achieved using, but is not limited to, the following methods: setting a learning rate of 0.01, an SGD optimizer, 200,000 iterations, and using label-smoothing to smooth labels to improve the generalization of the model. The loss function is optimized until the loss function no longer decreases or the total number of iterations is reached. Other restrictions are also possible, and are not limited in the specific embodiments of the present disclosure.

[0126] In some embodiments of the present disclosure, after the target image classification model is obtained through training, the use of the target image classification model can refer to the use of a normal deep learning model, and can be implemented by, but not limited to, the following methods: Figure 6 As shown, the image processing method includes:

[0127] Step 301: Input the image to be processed into the trained target image classification model for image processing to obtain the category of the image to be processed.

[0128] In the embodiment of the present disclosure, the image to be processed is input into the trained image classification model for image processing to obtain the category of the image to be processed. This can be achieved by, but is not limited to, the following method, which includes:

[0129] The image to be processed is input into the trained target image classification model to obtain a feature vector with a feature dimension of D. The image is classified based on the feature vector with a dimension of D to obtain the classification prediction probability. If the prediction probability is greater than the set threshold R for the class, it is judged to be of class c.

[0130] In an embodiment of the present disclosure, when image samples to be trained are received, data augmentation in different forms is performed twice on each image sample to be trained in the image samples to obtain augmented image samples to be trained; feature extraction is performed on the augmented image samples to be trained to obtain a feature vector corresponding to each augmented image sample to be trained, and a loss value is calculated based on a first loss function and the feature vector corresponding to the augmented image sample to be trained to obtain a first classification loss of the first augmented image sample to be trained, a second classification loss corresponding to the second augmented image sample to be trained, and a comparison loss corresponding to the first augmented image sample to be trained and the second augmented image sample to be trained; and based on the first classification loss, the second classification loss and the comparison loss, the sample type of each image sample to be trained is determined, and based on the curriculum learning algorithm, the image samples to be trained are iteratively trained on an initial image classification model in the order of the sample types to which they belong from simple, medium difficulty to difficult, to obtain a target image classification model. The training process of the entire image processing model is no longer limited to directly inputting the acquired image samples to be trained into the image classification model for training without distinction. Instead, based on the strongly supervised contrast loss and the self-supervised classification loss, the training image samples are classified into the types of training difficulty samples, and simple samples and difficult samples are distinguished. Based on the curriculum learning algorithm, the image samples to be trained are ordered from simple, medium difficulty to difficult in the order of their sample types, and the initial image classification model is iteratively trained. This solves the problem that the training effect of the image processing model is affected by noise samples, thereby improving the accuracy of image processing model training.

[0131] Based on the above Figures 1-6 A detailed description of the training method for the image processing model provided, such as Figure 7 As shown, Figure 6 FIG. 1 is a block diagram of a training device for an image processing model according to an exemplary embodiment. Figure 1 The device includes: a data processing unit 401, a feature extraction unit 402, a loss acquisition unit 403, a sample type determination unit 404 and a model training unit 405.

[0132] The data processing unit 401 is configured to perform two different forms of data augmentation on each of the training image samples to obtain augmented training image samples; the augmented training image samples include a first augmented training image sample and a second augmented training image sample;

[0133] A feature extraction unit 402 is configured to perform feature extraction on the augmented image samples to be trained, and obtain a feature vector corresponding to each augmented image sample to be trained;

[0134] a loss acquisition unit 403 configured to calculate a loss value based on a first loss function and a feature vector corresponding to the augmented image sample to be trained, to obtain, for each image sample to be trained, a first classification loss corresponding to the first augmented image sample to be trained, a second classification loss corresponding to the second augmented image sample to be trained, and a comparison loss corresponding to the first augmented image sample to be trained and the second augmented image sample to be trained;

[0135] a sample type determining unit 404 configured to determine the sample type of each to-be-trained image sample according to the first classification loss, the second classification loss, and the contrast loss;

[0136] The model training unit 405 is configured to iteratively train the initial image classification model based on the curriculum learning algorithm according to the order of sample types from simple, medium difficulty to difficult to obtain the target image classification model.

[0137] In some embodiments of the present disclosure, Figure 8 As shown, the loss acquisition unit 403 includes:

[0138] A first loss determination module 4031 is configured to calculate a classification loss value based on the first loss function and the feature vector corresponding to the augmented image sample to be trained, to obtain a first classification loss corresponding to the first augmented image sample to be trained and a second classification loss corresponding to the second augmented image sample to be trained for each image sample to be trained;

[0139] The second loss determination module 4032 is configured to calculate the distance between the first eigenvector and the second eigenvector corresponding to each image training sample to be trained, and obtain the comparison loss corresponding to the first augmented image sample to be trained and the second augmented image sample to be trained of each image sample to be trained, where the first eigenvector is the eigenvector corresponding to the first augmented image sample to be trained, and the second eigenvector is the eigenvector corresponding to the second augmented image sample to be trained.

[0140] In some embodiments of the present disclosure, the sample type determination unit 404 is configured to:

[0141] In response to the contrast loss being less than a first contrast loss threshold, and the first classification loss and the second classification loss being both less than the first classification loss threshold, determining that the to-be-trained image sample is a first simple sample, where the first simple sample represents an image sample that is correctly labeled and has a probability of being recognized greater than a first recognition threshold;

[0142] In response to the contrast loss being less than a first contrast loss threshold, and the first classification loss and the second classification loss being both greater than a second classification loss threshold, determining that the to-be-trained image sample is a second simple sample, where the second simple sample represents an image sample that is incorrectly labeled but has a probability of being recognized greater than the first recognition threshold;

[0143] In response to the contrast loss being greater than a second contrast loss threshold, and the first classification loss and the second classification loss having a classification loss greater than the second classification loss threshold and a classification loss less than the first classification loss threshold, determining that the to-be-trained sample is a medium-difficulty sample, the second classification loss threshold being greater than the first classification loss, and the medium-difficulty sample representing an image sample having a probability of being recognized greater than the second recognition threshold and less than the first recognition threshold;

[0144] In response to the contrast loss being greater than a second contrast loss threshold, and the first classification loss and the second classification loss being both greater than the second classification loss threshold, the image sample to be trained is determined to be a difficult sample, and the difficult sample represents an image sample whose probability of being recognized is less than a second recognition threshold.

[0145] In some embodiments of the present disclosure, the model training unit 405 is used to:

[0146] According to a first predetermined step length, the first simple sample is input into an initial image classification model for training to obtain a first image classification model;

[0147] According to a second predetermined step length, the first simple sample and the second simple sample are input into a first image classification model for training to obtain a second image classification model;

[0148] According to a third predetermined step length, the first simple sample, the second simple sample, and the moderately difficult sample are input into the second image classification model for training to obtain a third image classification model;

[0149] According to a fourth predetermined step length, the first simple sample, the second simple sample, the medium-difficulty sample, and the difficult sample are input into a third image classification model for training to obtain a target image classification model;

[0150] Wherein, the first predetermined step length, the second predetermined step length, the third predetermined step length, and the fourth predetermined step length decrease in sequence;

[0151] The sample weights of the first simple sample, the second simple sample, the medium-difficulty sample and the difficult sample are the largest during the first training and gradually decrease during the subsequent training process. The sum of the sample weights of each training of the image processing model is 1.

[0152] After the target image classification model is obtained through training, the use of the target image classification model can refer to the use of a normal deep learning model. Specifically, an image processing device is provided, such as Figure 9 As shown, the image processing device includes:

[0153] The image processing unit 501 is configured to input the image to be processed into the trained target image classification model for image processing to obtain the category of the image to be processed.

[0154] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0155] In an embodiment of the present disclosure, when image samples to be trained are received, data augmentation in different forms is performed twice on each image sample to be trained in the image samples to obtain augmented image samples to be trained; feature extraction is performed on the augmented image samples to be trained to obtain a feature vector corresponding to each augmented image sample to be trained, and a loss value is calculated based on a first loss function and the feature vector corresponding to the augmented image sample to be trained to obtain a first classification loss of the first augmented image sample to be trained, a second classification loss corresponding to the second augmented image sample to be trained, and a comparison loss corresponding to the first augmented image sample to be trained and the second augmented image sample to be trained; and based on the first classification loss, the second classification loss and the comparison loss, the sample type of each image sample to be trained is determined, and based on the curriculum learning algorithm, the image samples to be trained are iteratively trained on an initial image classification model in the order of the sample types to which they belong from simple, medium difficulty to difficult, to obtain a target image classification model. The training process of the entire image processing model is no longer limited to directly inputting the acquired image samples to be trained into the image classification model for training without distinction. Instead, based on the strongly supervised contrast loss and the self-supervised classification loss, the training image samples are classified into the types of training difficulty samples, and simple samples and difficult samples are distinguished. Based on the curriculum learning algorithm, the image samples to be trained are ordered from simple, medium difficulty to difficult in the order of their sample types, and the initial image classification model is iteratively trained. This solves the problem that the training effect of the image processing model is affected by noise samples, thereby improving the accuracy of image processing model training.

[0156] Figure 10 8 is a block diagram of an electronic device 800 shown in the present disclosure. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0157] Reference Figure 10, the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0158] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the information platform selection method described in any embodiment. In addition, the processing component 802 may include one or more modules to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

[0159] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0160] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0161] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0162] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0163] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0164] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the electronic device 800. The sensor assembly 814 can also detect changes in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and temperature changes of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0165] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, an operator network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0166] In an exemplary embodiment, the electronic device 800 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the information platform selection method described in any embodiment.

[0167] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is further provided, such as a memory 804 including instructions. The instructions may be executed by the processor 820 of the electronic device 800 to perform the information platform selection method described in any embodiment. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, or the like.

[0168] In an exemplary embodiment, a computer program product is also provided, comprising readable program code, which can be executed by the processor 820 of the apparatus 800 to implement the information platform selection method described in any embodiment. Optionally, the program code can be stored in a storage medium of the apparatus 800, which can be a non-transitory computer-readable storage medium, such as a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, or the like.

[0169] Figure 11 1 is a block diagram of an electronic device 1900 shown in the present disclosure. For example, the electronic device 1900 may be provided as a server.

[0170] Reference Figure 11The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the information platform selection method described in any embodiment.

[0171] The electronic device 1900 may further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0172] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0173] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A training method for an image processing model, characterized in that: include: Performing two different forms of data augmentation on each to-be-trained image sample in the to-be-trained image sample set to obtain augmented to-be-trained image samples; The augmented image samples to be trained include a first augmented image sample to be trained and a second augmented image sample to be trained; Extracting features from the augmented image samples to be trained, and obtaining a feature vector corresponding to each augmented image sample to be trained; Calculating a loss value according to a first loss function and a feature vector corresponding to the augmented image sample to be trained, to obtain, for each image sample to be trained, a first classification loss corresponding to the first augmented image sample to be trained, a second classification loss corresponding to the second augmented image sample to be trained, and a comparison loss corresponding to the first augmented image sample to be trained and the second augmented image sample to be trained; In response to the comparison loss being less than a first comparison loss threshold, and the first classification loss and the second classification loss being both less than the first classification loss threshold, determining that the to-be-trained image sample is a first simple sample, where the first simple sample represents an image sample that is correctly labeled and has a probability of being recognized greater than a first recognition threshold; In response to the comparison loss being less than a first comparison loss threshold, and the first classification loss and the second classification loss being both greater than a second classification loss threshold, determining that the to-be-trained image sample is a second simple sample, where the second simple sample represents an image sample that is incorrectly labeled but has a probability of being recognized greater than the first recognition threshold; In response to the comparison loss being greater than a second comparison loss threshold, and the first classification loss and the second classification loss having a classification loss greater than the second classification loss threshold and a classification loss less than the first classification loss threshold, determining that the to-be-trained image sample is a medium-difficulty sample, the second classification loss threshold being greater than the first classification loss, and the medium-difficulty sample representing an image sample having a probability of being recognized greater than a second recognition threshold and less than the first recognition threshold; In response to the comparison loss being greater than a second comparison loss threshold, and the first classification loss and the second classification loss being both greater than the second classification loss threshold, determining that the to-be-trained image sample is a difficult sample, where the difficult sample represents an image sample whose probability of being recognized is less than a second recognition threshold; Based on the curriculum learning algorithm, the image samples to be trained are sorted in order from simple, medium difficulty to difficult, and the initial image classification model is iteratively trained to obtain the target image classification model.

2. The method according to claim 1, characterized in that The calculating of the loss value according to the first loss function and the feature vector corresponding to the augmented image sample to be trained to obtain the first classification loss corresponding to the first augmented image sample to be trained, the second classification loss corresponding to the second augmented image sample to be trained, and the comparison loss corresponding to the first augmented image sample to be trained and the second augmented image sample to be trained for each image sample to be trained includes: Calculating a classification loss value according to the first loss function and the feature vector corresponding to the augmented image sample to be trained, to obtain a first classification loss corresponding to the first augmented image sample to be trained and a second classification loss corresponding to the second augmented image sample to be trained for each image sample to be trained; Calculate the distance between the first eigenvector and the second eigenvector corresponding to each image sample to be trained, and obtain the comparison loss corresponding to the first augmented image sample to be trained and the second augmented image sample to be trained for each image sample to be trained, where the first eigenvector is the eigenvector corresponding to the first augmented image sample to be trained, and the second eigenvector is the eigenvector corresponding to the second augmented image sample to be trained.

3. The method according to claim 1, characterized in that The curriculum learning algorithm is based on which the image samples to be trained are iteratively trained on the initial image classification model in the order of sample types from simple, medium difficulty to difficult to obtain the target image classification model, including: According to a first predetermined step length, the first simple sample is input into an initial image classification model for training to obtain a first image classification model; Inputting the first simple sample and the second simple sample into a first image classification model for training according to a second predetermined step length to obtain a second image classification model; According to a third predetermined step length, the first simple sample, the second simple sample, and the moderately difficult sample are input into a second image classification model for training to obtain a third image classification model; According to a fourth predetermined step length, the first simple sample, the second simple sample, the medium-difficulty sample, and the difficult sample are input into a third image classification model for training to obtain a target image classification model; Wherein, the first predetermined step length, the second predetermined step length, the third predetermined step length, and the fourth predetermined step length decrease in sequence; The sample weights of the first simple sample, the second simple sample, the medium-difficulty sample and the difficult sample are the largest during the first training and gradually decrease during the subsequent training process. The sum of the sample weights of each training of the image processing model is 1.

4. An image processing method, characterized in that: The method further includes: The image to be processed is input into the target image classification model trained by the training method of the image processing model in any one of claims 1 to 3 to perform image processing to obtain the category of the image to be processed.

5. A training device for an image processing model, characterized in that: include: A data processing unit is configured to perform two different forms of data augmentation on each to-be-trained image sample in the to-be-trained image samples to obtain augmented to-be-trained image samples; The augmented image samples to be trained include a first augmented image sample to be trained and a second augmented image sample to be trained; a feature extraction unit configured to perform feature extraction on the augmented image samples to be trained, and obtain a feature vector corresponding to each augmented image sample to be trained; a loss acquisition unit configured to calculate a loss value based on a first loss function and a feature vector corresponding to the augmented image sample to be trained, to obtain, for each image sample to be trained, a first classification loss corresponding to the first augmented image sample to be trained, a second classification loss corresponding to the second augmented image sample to be trained, and a comparison loss corresponding to the first augmented image sample to be trained and the second augmented image sample to be trained; a sample type determination unit, configured to determine the sample type of each to-be-trained image sample according to the first classification loss, the second classification loss, and the comparison loss; The model training unit is configured to iteratively train the initial image classification model based on the to-be-trained image samples in the order of sample types from simple, medium difficulty to difficult, based on a curriculum learning algorithm, to obtain a target image classification model; The sample type determination unit is specifically configured to: In response to the comparison loss being less than a first comparison loss threshold, and the first classification loss and the second classification loss being both less than the first classification loss threshold, determining that the to-be-trained image sample is a first simple sample, where the first simple sample represents an image sample that is correctly labeled and has a probability of being recognized greater than a first recognition threshold; In response to the comparison loss being less than a first comparison loss threshold, and the first classification loss and the second classification loss being both greater than a second classification loss threshold, determining that the to-be-trained image sample is a second simple sample, where the second simple sample represents an image sample that is incorrectly labeled but has a probability of being recognized greater than the first recognition threshold; In response to the comparison loss being greater than a second comparison loss threshold, and the first classification loss and the second classification loss having a classification loss greater than the second classification loss threshold and a classification loss less than the first classification loss threshold, determining that the to-be-trained image sample is a medium-difficulty sample, the second classification loss threshold being greater than the first classification loss, and the medium-difficulty sample representing an image sample having a probability of being recognized greater than a second recognition threshold and less than the first recognition threshold; In response to the comparison loss being greater than a second comparison loss threshold, and the first classification loss and the second classification loss being both greater than a second classification loss threshold, the image sample to be trained is determined to be a difficult sample, and the difficult sample represents an image sample whose probability of being recognized is less than a second recognition threshold.

6. The device according to claim 5, characterized in that The loss acquisition unit is configured to: a first loss determination module configured to calculate a classification loss value according to the first loss function and the feature vector corresponding to the augmented image sample to be trained, to obtain a first classification loss corresponding to the first augmented image sample to be trained and a second classification loss corresponding to the second augmented image sample to be trained for each image sample to be trained; The second loss determination module is configured to calculate the distance between the first eigenvector and the second eigenvector corresponding to each image training sample to be trained, and obtain the comparison loss corresponding to the first augmented image sample to be trained and the second augmented image sample to be trained of each image sample to be trained, where the first eigenvector is the eigenvector corresponding to the first augmented image sample to be trained, and the second eigenvector is the eigenvector corresponding to the second augmented image sample to be trained.

7. An image processing device, characterized in that include: The image processing unit is configured to input the image to be processed into the target image classification model trained by the training method of the image processing model according to any one of claims 1 to 3 to perform image processing to obtain the category of the image to be processed.

8. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the training processing method of the image processing model as described in any one of claims 1 to 3 or the image processing method as described in claim 4.

9. A computer-readable storage medium, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to execute the training method of the image processing model as described in any one of claims 1 to 3 or the image processing method as described in claim 4.

Images