Image content classification method, apparatus and device, and storage medium and product

Abstract

An image content classification method, apparatus, and device, and a storage medium and a product. The method comprises: acquiring an image to be subjected to recognition (S110); and inputting said image into a trained image content classification model, and performing analysis processing on said image by means of the image content classification model, so as to obtain a target label of said image (S120), wherein the image content classification model is obtained by means of performing training on the basis of a set loss function, a sample image and a potential noisy image, the potential noisy image being obtained by means of performing data augmentation and noisy sample recognition on the sample image, and the loss function being determined on the basis of the frequency at which the potential noisy image is recognized as a potential noisy image. A potential noisy sample is recognized and retained, and an image content classification model is trained on the basis of a dynamically determined loss function, such that while the negative impact of a noisy sample during training is reduced, an indistinguishable hard sample can be retained to participate in training, so as to improve the recognition performance of the model for a hard sample, thereby effectively improving the accuracy of image content classification.

Description

Image content classification method, device, equipment, storage medium and product

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on March 1, 2024, with application number 202410234378.1, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of image processing technology, and in particular to an image content classification method, apparatus, device, storage medium, and product. Background Art

[0003] Image content classification is a fundamental problem in computer vision, with extensive research in academia and widespread application in industry, such as image content review and understanding. Most related technologies for image classification are based on statistical machine learning and deep neural networks. The basic process involves predefining a set of target categories and collecting labeled training data. The model and loss function are then selected based on the task objectives. Model parameters are then updated using optimization algorithms such as gradient descent to minimize the predicted category error, resulting in a model capable of classifying image content according to the target category.

[0004] In practical applications, label noise often becomes a performance bottleneck for image classification models. The category labels of training data usually come directly or indirectly from manual annotation. However, in most application scenarios, manual annotation cannot guarantee 100% accuracy. That is, some images (i.e., noise samples) are labeled with incorrect category labels. These incorrect category labels will form noise interference in the subsequent image classification model learning.

[0005] Related technologies typically process noise samples by using methods such as multiple model ensembles and feature clustering to identify noise samples with labeled noise, and then directly filter out the noise samples. However, this model training approach causes a large number of non-noise samples to be treated as noise samples, affecting data utilization efficiency during model training and even negatively impacting final model performance, reducing the accuracy of image content classification. Summary of the Invention

[0006] The embodiments of the present application provide an image content classification method, apparatus, device, storage medium, and product to solve the technical problem of low image content classification accuracy under the model training method of directly filtering noise samples in the related art, and effectively improve the image content classification accuracy.

[0007] In a first aspect, an embodiment of the present application provides an image content classification method, comprising:

[0008] Obtain the image to be recognized;

[0009] The image to be identified is input into a trained image content classification model, and the image to be identified is analyzed and processed by the image content classification model to obtain a target label of the image to be identified. The image content classification model is trained based on a set loss function according to sample images and potential noise images. The potential noise images are obtained by performing data enhancement and noise sample identification on the sample images. The loss function is determined according to the frequency with which the potential noise image is identified as a potential noise image.

[0010] In a second aspect, an embodiment of the present application provides an image content classification device, including an image acquisition module and an image classification module, wherein:

[0011] The image acquisition module is configured to acquire an image to be identified;

[0012] The image classification module is configured to input the image to be identified into a trained image content classification model, analyze and process the image to be identified through the image content classification model, and obtain a target label for the image to be identified. The image content classification model is trained based on a set loss function according to sample images and potential noise images. The potential noise images are obtained by performing data enhancement and noise sample identification on the sample images. The loss function is determined based on the frequency with which the potential noise image is identified as a potential noise image.

[0013] In a third aspect, an embodiment of the present application provides an image content classification device, including: a memory and one or more processors;

[0014] The memory is used to store one or more programs;

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the image content classification method as described in the first aspect.

[0016] In a fourth aspect, an embodiment of the present application provides a non-volatile storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are used to perform the image content classification method as described in the first aspect.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor of the device reads and executes the computer program from the computer-readable storage medium, so that the device performs the image content classification method described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] FIG1 is a flow chart of an image content classification method provided by an embodiment of the present application;

[0019] FIG2 is a schematic diagram of the training process of an image content classification model;

[0020] FIG3 is a schematic diagram of a first noise image determination process according to an embodiment of the present application;

[0021] FIG4 is a schematic diagram of a second noise image determination process according to an embodiment of the present application;

[0022] FIG5 is a schematic diagram of a second loss function determination process provided in an embodiment of the present application;

[0023] FIG6 is a schematic diagram of the structure of an image content classification device provided in an embodiment of the present application;

[0024] FIG7 is a schematic structural diagram of an image content classification device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. It should also be noted that, for ease of description, only some, but not all, of the contents related to the present application are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The above process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The above process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0026] The image content classification method provided in this application can be applied to the classification and identification of image content in short video and graphic information flow business scenarios. It aims to analyze and process the image to be identified through an image content classification model to obtain a target label for the image to be identified. The image content classification model is based on a set loss function and is trained based on sample images and potential noise images. The potential noise image is obtained by performing data enhancement on the sample image and identifying noise samples. The loss function is determined according to the frequency with which the potential noise image is identified as a potential noise image. It can efficiently handle the label noise problem in image content classification and is widely applicable to different image content classification tasks.

[0027] Existing image content classification schemes are mostly based on statistical machine learning and deep neural networks. They collect a batch of labeled training data for the target task and category, then update the model parameters on this training data in a way that minimizes a loss function. Ultimately, an image classification model is obtained that maps the input image to one or more categories in the target category set. In practical applications, label noise is a significant factor affecting the performance of image content classification models. Because image classification models based on deep neural networks typically have a large number of learnable parameters and strong fitting capabilities, data with label noise can easily be misfitted during the model learning process, causing the model to learn incorrect visual features and mapping relationships, resulting in a high incidence of incorrect category mappings in practical applications.

[0028] There are several common types of label noise: 1. Mislabeling in single-label scenarios, where images matching category A are mistakenly labeled as category B when each image matches only one label in the target label set; 2. Mislabeling in multi-label scenarios, where each image can match any number of labels (including zero) in the target label set, but the image content does not match a specific category A but is mistakenly labeled as category A; and 3. Omission in multi-label scenarios, where the image content matches a specific category A but is not labeled as category A. Types 1 and 2 of mislabeling can be categorized as obvious mislabeling (the image content and the mislabeled category are completely unrelated), hard mislabeling (the image content and the mislabeled category have a certain visual similarity but can be distinguished with careful observation), and ambiguous mislabeling (it is impossible to accurately determine whether the ambiguous image content matches the mislabeled category) based on the degree of match between the image content and the mislabeled category. Different types of label noise can have different negative impacts on the learning of image content classification models. In practical applications, multiple types of label noise often coexist, further exacerbating the impact of label noise on model performance.

[0029] To combat label noise, methods for identifying and processing noisy samples are commonly employed. For example, methods such as multi-model ensembles, multi-model parallel training, feature matching, and clustering are used to identify potential noisy samples in the training data. These samples (those suspected to be noisy) are then discarded. Alternatively, methods such as semi-supervised learning are used to combine the samples themselves (i.e., images) with the remaining labeled data and participate in model learning. These methods often involve multi-stage offline processing, such as pre-training the model, screening for noisy samples, and retraining and fine-tuning the model. This multi-stage process results in high time and computational overhead in practical use, hindering model development efficiency. Furthermore, these methods still rely on the model itself and its extracted image features to identify noisy samples. In complex scenarios and tasks, they often fail to effectively distinguish between noisy and difficult samples. Discarding difficult samples along with noise, or treating them as unlabeled samples, impairs the model's ability to identify difficult samples.

[0030] Based on this, an image content classification method according to an embodiment of the present application is provided to solve the technical problem of low image content classification accuracy under the model training method of directly filtering noise samples in existing image content classification schemes.

[0031] Figure 1 shows a flowchart of an image content classification method provided in an embodiment of the present application. The image content classification method provided in an embodiment of the present application can be executed by an image content classification device, which can be implemented by hardware and / or software and integrated into an image content classification device (such as a server).

[0032] The following description is based on an example of an image content classification method executed by an image content classification device. Referring to FIG1 , the image content classification method includes:

[0033] S110: Acquire an image to be recognized.

[0034] For example, when it is necessary to perform content classification and recognition on images of information such as short videos, graphic information streams, etc., the corresponding images can be obtained as images to be recognized.

[0035] For example, when it is necessary to conduct content review, content tagging, image content understanding, etc. on short videos or graphic information streams based on the labels corresponding to the images, the corresponding images can be obtained as the images to be identified.

[0036] S120: Input the image to be identified into the trained image content classification model, analyze and process the image to be identified through the image content classification model, and obtain a target label for the image to be identified.

[0037] Among them, the image content classification model is based on a set loss function and is trained according to sample images and potential noise images. The potential noise images are obtained by data enhancement of the sample images and noise sample identification. The loss function is determined according to the frequency with which the potential noise image is identified as a potential noise image.

[0038] This solution uses a trained image content classification model to perform image content classification and recognition on the image to be recognized. Among them, the image content classification model provided by this solution is obtained by training based on sample images and potential noise images based on a set loss function. And the potential noise image is obtained by performing data enhancement and noise sample recognition on the sample image, and the loss function is determined according to the frequency with which the potential noise image is recognized as a potential noise image. Among them, by identifying and retaining potential noise samples and training the image content classification model based on a dynamically determined loss function, the negative impact of noise samples during training can be effectively reduced while retaining difficult samples that are difficult to distinguish to participate in training, thereby improving the model's recognition effect on difficult samples. The different loss functions provided by this solution (including the first to seventh loss functions) can be the same or different. The loss function can be a combination of one or more of absolute value loss, square loss, logarithmic loss, Hinge loss, exponential loss and 0-1 loss.

[0039] After the image content classification model is trained, the trained image content classification model can be used. For example, the image to be identified is input into the image content classification model, where it is used to perform image content classification and identification on the image to be identified, obtaining a target label corresponding to the image to be identified. After post-processing the target label, such as label mapping, the result can be output to downstream business logic for use in scenarios such as content filtering, content screening, and content recommendation.

[0040] During the training process of conventional image content classification models, periodic model inference with data augmentation is typically performed to identify potential noise samples based on the error between the model score and the image class label. To address the problems of model overfitting to noise samples during training and the inability to distinguish between noise and difficult samples in complex scenarios, this solution applies multiple different data augmentations to the same sample image and improves the accuracy of noise sample identification based on the inference results of these enhanced images. After identifying potential noise samples, most related technologies discard or automatically modify their labels. However, in real-world scenarios, identified potential noise samples almost inevitably contain some clean and difficult samples. Automatically modifying labels will artificially introduce additional noise samples, while directly discarding them fails to effectively utilize difficult samples to improve model performance. This solution addresses potential noise samples by adding dynamic margin to the loss function. This approach not only reduces the negative impact of noise samples on model training, but also allows the difficult samples they contain to continue participating in training. This approach improves model recognition accuracy without compromising the model's performance on difficult samples.

[0041] In one embodiment, the training of the image content classification model can be divided into a conventional training process based on sample images and a fixed loss function (training the initial image content classification model to obtain an intermediate image content classification model), and an enhanced training process based on sample images and potentially noisy images, as well as a fixed loss function and a dynamic loss function (training the intermediate image content classification model to obtain the final image content classification model). This can effectively reduce the negative impact of label noise on the image content classification model in different business scenarios and tasks. Optionally, the image content classification model provided by this solution can be built based on a set neural network structure (such as a convolutional neural network CNN, a Vision Transformer, etc.).

[0042] In one possible embodiment, as shown in FIG2 , a training process diagram of an image content classification model is provided. The training process of the image content classification model provided by this solution includes:

[0043] S101: Performing a first training on an initial image content classification model according to a sample image and a first loss function to obtain an intermediate image content classification model.

[0044] The sample images provided in this solution can be collected and processed based on images corresponding to a predetermined target application scenario, each of which corresponds to at least one category label. For example, an initial image content classification model can be first trained based on the sample images and a first loss function for a predetermined number of training rounds to obtain an intermediate image content classification model.

[0045] Among them, the first training of the initial image content classification model is a multi-round iterative training process. All N sample images D = {(x1, y1), (x2, y2), ..., (xN, yN)} in the sample image set will be traversed for M rounds (epochs). In each round, there will be multiple iterations of batch data (databatch). Each iteration samples S sample images from the sample image set to form a batch data (batch): B = {(x1, y1), (x2, y2), ..., (xS, yS)}, where xi is the sample image i and yi is the category label corresponding to the sample image i. S sample images are respectively input into the initial image content classification model to obtain the category prediction results P = {p1, p2, ..., pS} of the initial image content classification model, and the loss function L = {h(p1, y1), h(p2, y2), ..., h(pS, yS)} of each sample image is calculated based on the set first loss function (loss function) h(·). Then, by setting an optimization algorithm (such as gradient descent optimization algorithm), the model parameters of the initial image content classification model are updated with the goal of minimizing the loss function to obtain an intermediate image content classification model.

[0046] S102: performing enhancement processing on the sample image to obtain an enhanced image, and performing noise sample identification on the enhanced image to obtain a potential noise image.

[0047] Exemplarily, data enhancement processing is performed on each of the above sample images based on a set data enhancement method to obtain multiple enhanced images. The data enhancement method provided in this solution can be a combination of one or more of image flipping, image rotation, image scaling, image translation, image cropping, color enhancement, noise enhancement, filter enhancement, and data expansion. Optionally, each sample image can be randomly data enhanced to obtain multiple enhanced images, and data enhancement processing can also be performed according to multiple different data enhancement methods and / or a combination of multiple different data enhancement methods to obtain multiple enhanced images.

[0048] In one embodiment, after obtaining multiple enhanced images, noise sample identification is performed on the multiple enhanced images, and multiple potential noise images are determined from the multiple enhanced images based on the noise sample identification results. Alternatively, noise sample identification can be performed on the enhanced images and sample images to obtain multiple potential noise images from the enhanced images and sample images.

[0049] The potential noise image can be understood as an enhanced image that may be a noise sample. Optionally, a first noise image can be determined from the enhanced image based on the score consistency between the class label of the sample image and the model score of the enhanced image (i.e., the predicted label output by the intermediate image content classification model), and / or a second noise image can be determined from the enhanced image based on the difference in prediction error distribution between the sample image and the enhanced image.

[0050] In one embodiment, the second training of the intermediate image content classification model will be carried out for multiple rounds. In each round of the second training, the noise sample identification of the enhanced image will be re-performed to obtain the potential noise image of the corresponding training round. As the number of training rounds of the second training increases, the accuracy of the determined potential noise image as a noise sample gradually increases.

[0051] Since the model will overfit to noise samples during training, it is difficult to effectively distinguish between clean samples and noise samples based solely on the model's prediction error for each training sample. Low noise sample recognition accuracy and missed recognition often occur. Related technologies generally address this issue through multi-model integration or feature matching. The former requires training multiple models simultaneously, which significantly increases training overhead, while the latter relies on a feature extraction model that is insensitive to noise and lacks feasibility in practical scenarios. This solution uses multiple random data augmentations to address this problem. Data augmentation in the inference phase (test-time augmentation) will be used to improve model evaluation results in dataset evaluation. Compared with random data augmentation in the training phase, data augmentation in the inference phase usually uses a set of predefined deterministic data augmentation strategies. During the inference phase, this scheme uses a set of predefined data augmentation strategies A = {A1, A2, ..., Am}. Each sample image x is subjected to k random data augmentations, resulting in k enhanced images X = {x1, x2, ..., xk}. These images are then fed into the intermediate image content classification model of the current training phase, yielding k sets of model scores P = {p1, p2, ..., pk}. For clean samples without label noise, the model scores of different augmented samples are generally highly consistent and have a small error relative to the given class label. However, for noisy samples, random data augmentation reduces the impact of overfitting, resulting in lower consistency in model scores across augmented samples. Based on the differences between the augmented images obtained after data augmentation and the sample images, the augmented images can be used to identify noisy samples and identify potential noisy images.

[0052] In one possible embodiment, when noise samples are identified on an enhanced image to obtain a potential noise image, the following steps may be performed: determining a first noise image from the enhanced image based on the score consistency between the sample image and the enhanced image; and / or determining a second noise image from the enhanced image based on the prediction error distribution between the sample image and the enhanced image.

[0053] Exemplarily, for determining the first noise image based on score consistency, for each enhanced image, the score consistency between the enhanced image and the corresponding sample image (the sample image corresponds to the enhanced image obtained by data augmentation based on the sample image) is determined. The score consistency between the enhanced image and the corresponding sample image can be understood as the degree of consistency between the model score obtained by inputting the enhanced image into the intermediate image content classification model and the classification label of the corresponding sample image. Based on the score consistency between the sample image and the enhanced image, the first noise image is determined from the enhanced image. For example, the enhanced image whose score consistency reaches a set threshold is used as the first noise image. The first noise image can be used as a potential noise image for the second training of the intermediate image content classification model. This solution accurately identifies potential noise samples and improves the accuracy of image content classification by determining the first noise image as a potential noise image from the enhanced image based on the score consistency between the sample image and the enhanced image.

[0054] Exemplarily, to determine the second noise image based on the prediction error distribution, the prediction error between each enhanced image model score and the classification label corresponding to the sample image is determined, and the second noise image is determined from the enhanced image based on the distribution of prediction errors corresponding to multiple enhanced images. This solution accurately identifies potential noise samples and improves image content classification accuracy by determining the second noise image from the enhanced image as a potential noise image based on the prediction error distribution between the sample image and the enhanced image.

[0055] In one possible embodiment, as shown in a schematic diagram of a first noisy image determination process provided in FIG3 , the image content classification method provided in this solution, when determining the first noisy image from the enhanced image based on the score consistency between the sample image and the enhanced image, includes:

[0056] S1021: Determine the model score of each enhanced image through the intermediate image content classification model.

[0057] Exemplarily, for each enhanced image, the enhanced image is input into the intermediate image content classification model obtained through the first training, and the enhanced image is analyzed and processed by the intermediate image content classification model to obtain a model score corresponding to the enhanced image.

[0058] Among them, the model score (such as the predicted label) output by the image content classification model (including the initial, intermediate and final models) has the same dimension as the classification label corresponding to the sample image, for example, recording the probabilities or scores of multiple set classifications.

[0059] S1022: Determine the noise index of each enhanced image based on the model score and the category label corresponding to the sample image.

[0060] Exemplarily, for each enhanced image, the noise index of the enhanced image is determined based on the model score corresponding to the enhanced image and the category label of the corresponding sample image. For example, the noise index is determined based on the error between the model score corresponding to the enhanced image and the category label of the corresponding sample image. The greater the error between the model score and the category label, the greater the corresponding noise index.

[0061] In one possible embodiment, the image content classification method provided by the present solution, when determining the noise index of each enhanced image based on the model score and the category label corresponding to the sample image, includes: determining the first sample error and sample standard deviation between the model score and the category label corresponding to the sample image; determining the noise index of each enhanced image based on the first sample error and the sample standard deviation.

[0062] Optionally, the errors provided by this scheme (including prediction error, first prediction error, second prediction error, first sample error, second sample error and third sample error) can be determined by a combination of one or more of absolute error, mean square error, hinge loss error and cross entropy error.

[0063] Exemplarily, a first sample error and a sample standard deviation between the model score of the enhanced image and the category label of the corresponding sample image are determined, wherein the first sample error can be the error between the average model score and the category label, and the sample standard deviation can be the standard deviation corresponding to the error between the model score and the category label.

[0064] After determining the first sample error and sample standard deviation corresponding to each enhanced image, a noise index for each enhanced image is determined based on the first sample error and sample standard deviation. For example, the noise index for the enhanced image can be determined by taking the sum of the first sample error and sample standard deviation of each enhanced image, or by taking the weighted sum of the first sample error and sample standard deviation of each enhanced image, or by taking the product of the first sample error and sample standard deviation of each enhanced image. This solution improves the accuracy of determining the first noise image, accurately identifies potential noise samples, and improves the accuracy of image content classification by determining the noise index for each enhanced image based on the first sample error and sample standard deviation corresponding to each enhanced image.

[0065] In one possible embodiment, the image content classification method provided by the present solution, when determining the first sample error and sample standard deviation between the model score and the category label corresponding to the sample image, includes: determining the first sample error between the average model score of each enhanced image and the category label corresponding to the sample image based on a set third loss function; determining the second sample error between the model score of each enhanced image and the category label corresponding to the sample image based on a set fourth loss function, and determining the sample standard deviation between the model score and the category label corresponding to the sample image based on the standard deviation of the second sample error.

[0066] Exemplarily, an average model score of each enhanced image is determined, and a first sample error between the average model score of each enhanced image and the category label of the corresponding sample image is determined based on a set third loss function. The average model score is the average score of each dimension in the model score.

[0067] In one embodiment, based on a set fourth loss function, the second sample error between the model score of each enhanced image and the category label corresponding to the sample image is determined, and the standard deviation corresponding to the second sample error of each enhanced image is determined. The model score of the enhanced image and the sample standard deviation between the category labels corresponding to the sample image are determined based on the standard deviation of the multiple second sample errors. This solution effectively improves the accuracy of determining the first noise image, accurately identifies potential noise samples, and improves the accuracy of image content classification by determining the first sample error of each enhanced image based on a set third loss function, and the second sample error of each enhanced image based on a set fourth loss function, and determining the model score and the sample standard deviation between the category labels corresponding to the sample image based on the standard deviation of the second sample error.

[0068] S1023: Determine a first noise image from the enhanced image according to the noise index.

[0069] Exemplarily, multiple first noise images are determined from multiple enhanced images based on the noise index of each enhanced image. For example, an enhanced image whose noise index reaches a set threshold can be determined as the first noise image, or a set first ratio or number of enhanced images ranked first can be determined as the first noise image. This solution determines the noise index of each enhanced image based on the model score and the category label corresponding to the sample image, and determines the first noise image from the enhanced images as a potential noise image based on the noise index, accurately identifying potential noise samples and improving the accuracy of image content classification.

[0070] In a possible embodiment, the image content classification method provided by the present solution, wherein determining the first noise image from the enhanced image according to the noise index includes: determining the first noise image from the enhanced image according to the sorting of the noise index and a set first ratio.

[0071] For example, the noise indicators corresponding to the enhanced images are sorted from largest to smallest, and the enhanced images with the highest noise indicators ranked first by a predetermined first ratio are determined as first noise images. This solution accurately identifies potential noise samples and improves the accuracy of image content classification by determining the first noise image from the enhanced images based on the sorting of noise indicators and the predetermined first ratio.

[0072] For example, the fourth loss function is first used to calculate the set of second sample errors E = {h(p1, y), h(p2, y), ..., h(pk, y)} between the k enhanced images and the class labels, as well as the average model score of the k enhanced images and the first sample error between the class labels The sample standard deviation σ of the second sample error set E is calculated. Optionally, the sum of the first sample error e and the sample standard deviation σ can be used as the noise index of the enhanced image: n = e + σ. The above process is performed on all N sample images to obtain a set of noise indexes N = {n1, n2, ..., nN}. The noise index set N is sorted from high to low, and the first portion of enhanced images (e.g., the multiple enhanced images before the set first ratio r1) are used to obtain multiple first noise images (potential noise sample set U).

[0073] In one possible embodiment, as shown in a schematic diagram of a second noise image determination process provided in FIG4 , the image content classification method provided in this solution, when determining the second noise image from the enhanced image based on the prediction error distribution between the sample image and the enhanced image, includes:

[0074] S1024: Determine a first prediction error distribution of the sample image and a second prediction error distribution of the enhanced image.

[0075] Exemplarily, based on the multiple sample images and enhanced images determined above, the first prediction error distribution of the sample images and the second prediction error distribution of the enhanced images are fitted based on a set data distribution model. In one embodiment, the first prediction error distribution of the sample images and the second prediction error distribution of the enhanced images can be fitted based on a Gaussian Mixture Model (GMM).

[0076] Among them, assuming that the prediction error distributions of the clean samples and noise samples in the sample images follow different statistical laws after data augmentation, the prediction error of the clean samples follows the first prediction error distribution Ω1, and the prediction error of the noise samples follows the second prediction error distribution Ω2. The first prediction error distribution Ω1 and the second prediction error distribution Ω2 can be fitted from the prediction errors {E1, E2, …, EN} in the sample images and enhanced images, and based on the maximum likelihood principle, it is determined whether each sample (sample image and enhanced image) belongs to a noise sample (second noise image). To fit the first prediction error distribution Ω1 and the second prediction error distribution Ω2, a Gaussian mixture distribution model can be used for fitting. This Gaussian mixture distribution model assumes that the overall distribution of the samples is composed of the weighted sum of g Gaussian distributions, and the sum of the g weights is 1. The fitting of the Gaussian mixture distribution model includes two steps: estimating the weight coefficients of the g Gaussian distributions and estimating the mean and covariance of each Gaussian distribution, which can be iteratively completed using the Expectation Maximization (EM) algorithm. Assuming that the prediction errors of the clean samples and noise samples follow different distributions respectively, so g = 2 in the Gaussian mixture distribution model. Based on the specific business scenario and data, it can be further assumed that the proportion of noise samples does not exceed a set second proportion r2 (0 < r2 < 1), corresponding to restricting the weight coefficient of the noise sample distribution in the Gaussian mixture distribution model to be no greater than r2. Based on the Gaussian mixture distribution model, the first prediction error distribution Ω1 of the clean samples and the second prediction error distribution Ω2 of the noise samples can be fitted from all the training data.

[0077] S1025: Determine the likelihood ratio of the second sample error corresponding to each enhanced image following the first prediction error distribution and the second prediction error distribution.

[0078] Illustratively, based on the first prediction error distribution and the second prediction error distribution determined above, a first likelihood value for the second sample error corresponding to each enhanced image to follow the first prediction error distribution and a second likelihood value for the second sample error corresponding to each enhanced image to follow the second prediction error distribution are determined, and a likelihood ratio of the second sample error corresponding to each enhanced image to follow the first prediction error distribution and the second prediction error distribution is determined based on the second likelihood value and the first likelihood value. For example, the ratio of the second likelihood value to the first likelihood value can be determined as the corresponding likelihood ratio.

[0079] For example, for each sample image, the second sample error E = {h(p1, y), h(p2, y), ..., h(pk, y),} corresponding to the k enhanced images, the likelihood ratios of their compliance with Ω1 and Ω2 are calculated respectively: For the mixed Gaussian distribution model, Ω1 and Ω2 are both Gaussian distributions, and their likelihood function is Where μ and σ are the covariances of the mean of the Gaussian distribution, which can be obtained by fitting the expected value maximization algorithm.

[0080] S1026: Determine a second noise image from the enhanced image according to the likelihood ratio.

[0081] Exemplarily, for each enhanced image, whether to determine the enhanced image as the second noise image is determined according to the likelihood ratio, for example, the enhanced image whose likelihood ratio reaches a set likelihood ratio threshold is determined as the second noise image, thereby determining multiple second noise images from multiple enhanced images.

[0082] Among them, the likelihood ratio l k A value greater than 1 indicates that the enhanced image is more likely to be a noise sample. For each sample image, the average likelihood ratio of its k enhanced images is calculated: The set of enhanced images with an average likelihood ratio greater than 1 is the set of the second noise image (potential noise sample set V). Alternatively, in order to reduce computational complexity and improve numerical stability, the logarithmic form of the likelihood ratio can also be taken. In this case, l j The set of training samples whose average value l is greater than 0 is the set of second noise images. This solution accurately determines the second noise image from the enhanced image based on the likelihood ratio of the second sample error corresponding to the enhanced image to the first prediction error distribution and the second prediction error distribution, accurately identifying potential noise samples and improving the accuracy of image content classification.

[0083] In one possible embodiment, when noise samples are identified on an enhanced image to obtain a potential noise image, the following steps may be performed: based on the score consistency between the sample image and the enhanced image, a first noise image is determined from the enhanced image; based on the prediction error distribution between the sample image and the enhanced image, a second noise image is determined from the enhanced image; and the potential noise image is determined based on the intersection of each first noise image and the second noise image.

[0084] Among them, based on the score consistency between the sample image and the enhanced image, the relevant description of the first noise image is determined from the enhanced image, and the above-mentioned determination steps of the first noise image can be referred to. Based on the prediction error distribution between the sample image and the enhanced image, the relevant description of the second noise image is determined from the enhanced image, and the above-mentioned determination steps of the second noise image can be referred to, and the corresponding technical effects are achieved, which will not be repeated in this solution.

[0085] After determining multiple first noise images and multiple second noise images, the intersection of the first noise images and the second noise images is taken, and the intersection is used as the set of potential noise images. For example, the set U of first noise images and the set V of second noise images are intersected to obtain the final set of potential noise images N = U ∩ V. This solution determines the potential noise image by taking the intersection of the first noise image and the second noise image obtained based on score consistency and prediction error distribution. By taking the intersection of potential noise images obtained using different strategies, it reduces the misidentification of noise samples caused by a single recognition strategy, reduces the number of clean samples and difficult samples mistaken for noise samples, effectively improves the recognition accuracy of noise samples, and requires fewer hyperparameters to be selected and adjusted based on prior knowledge. Among them, the hyperparameters that need to be selected are the first ratio r1 (potential noise sample ratio) and the second ratio r2 (the maximum coefficient of the noise sample distribution in the Gaussian distribution model). For example, it can be set to r1 = r2, that is, only one hyperparameter needs to be selected and adjusted according to the specific task and scenario. For data augmentation strategy A, you can choose according to the specific task and scenario, such as using random cropping data augmentation, random flipping data augmentation, random blurring / sharpening data augmentation, etc.

[0086] S103: Determine a second loss function of the potential noise image according to the sample image, the potential noise image, and the frequency at which the potential noise image is identified as a potential noise image.

[0087] Exemplarily, the frequency with which each potential noise image is identified as a potential noise image is determined. Different enhanced images may be identified as potential noise images in different training rounds, and the frequency with which a potential noise image is identified as a potential noise image is the frequency with which the image was identified as a potential noise image in the current or previous training round.

[0088] In one embodiment, a second loss function is determined for each potential noise image based on a plurality of sample images, a plurality of potential noise images, and the determined frequency of each potential noise image being identified as a potential noise image. For example, for each potential noise image, a sample error between the class label of the potential noise image and the corresponding sample image (i.e., the sample image from which the corresponding enhanced image was generated) is determined, and the second loss function for the potential noise image is determined based on the sample error and the frequency of each potential noise image being identified as a potential noise image.

[0089] In one possible embodiment, as shown in a schematic diagram of a second loss function determination process provided in FIG5 , the image content classification method provided in this solution, when determining the second loss function of the potential noise image based on the sample image, the potential noise image, and the frequency with which the potential noise image is identified as a potential noise image, includes:

[0090] S1031: Based on the set fifth loss function, determine the third sample error between the model score of each potential noise image and the category label corresponding to the sample image.

[0091] S1032: Determine a second loss function of the potential noise image according to the third sample error and the frequency at which the potential noise image is identified as a potential noise image.

[0092] Exemplarily, for each potential noise image, a third sample error between the model score of each potential noise image and the category label of the corresponding sample image is determined based on the set fifth loss function.

[0093] Determine the frequency with which the potential noise image is identified as a potential noise image in the current second training round and the previous second training round (i.e., the frequency with which the enhanced images corresponding to each current second training round enter the set of potential noise images). Determine the second loss function of the potential noise image based on the third sample error and the frequency with which the potential noise image is identified as a potential noise image. This solution accurately determines the second loss function of the potential noise image based on the third sample error of the potential noise image and the frequency with which the potential noise image is identified as a potential noise image, dynamically adjusts the loss function of suspected noise samples, improves the model's recognition effect on difficult samples, and effectively improves the accuracy of image content classification.

[0094] In one possible embodiment, the image content classification method provided by the present solution, when determining the second loss function of the potential noise image based on the third sample error and the frequency at which the potential noise image is identified as a potential noise image, may be: determining the dynamic interval of each potential noise image based on the third sample error and the frequency at which the potential noise image is identified as a potential noise image; and determining the second loss function of the potential noise image based on the dynamic interval and the third sample error.

[0095] Exemplarily, a dynamic margin of each potential noise image is determined based on the third sample error determined above and the frequency at which the potential noise image is identified as a potential noise image. For example, the dynamic margin m can be determined based on the third sample error e(p,y) of the potential noise image and the frequency f at which the potential noise image is identified as a potential noise image: m = f*∑ max e(p,y), where max refers to the maximum value of the third sample error.

[0096] In one embodiment, for each potential noise image, a second loss function of the potential noise image can be determined based on the corresponding dynamic interval and the third sample error. For example, the second loss function can be determined from the set loss function based on a comparison of the dynamic interval and the third sample error.

[0097] For example, when the comparison between the dynamic interval and the third sample error indicates that the probability of the potential noise image being a noise sample is large or reaches a set probability threshold, the second loss function is determined to be the set valid loss function (such as the first loss function), and the potential noise image is retained for the second training. When the comparison between the dynamic interval and the third sample error indicates that the probability of the potential noise image being a noise sample is small or is less than the set probability threshold, the second loss function is determined to be the set invalid loss function (such as 0), which is equivalent to discarding the potential noise image and not participating in the second training. This scheme accurately determines the second loss function of the potential noise image based on the dynamic interval and the third sample error of each potential noise image. The determination of the dynamic interval can make the loss function of data with a greater probability of being a noise sample more likely to be zero (higher frequency in the set N, larger dynamic interval m), while the dynamic interval corresponding to the difficult samples that are occasionally mistakenly identified as noise samples is smaller, and has less impact on its loss function. This can not only reduce the impact of noise samples, but also use difficult samples mixed in noise samples for training, effectively improving the model's recognition effect on difficult samples.

[0098] In one possible embodiment, the image content classification method provided by the present solution, when determining the second loss function of the potential noise image based on the dynamic interval and the third sample error, may be: when the third sample error is greater than or equal to the dynamic interval, the set sixth loss function is determined as the second loss function of the potential noise image; when the third sample error is less than the dynamic interval, the set seventh loss function is determined as the second loss function of the potential noise image.

[0099] Exemplarily, after determining the dynamic interval of the potential noise image, the third sample error is compared with the dynamic interval. When the third sample error is greater than or equal to the dynamic interval, a set sixth loss function (e.g., the first loss function) may be determined as the second loss function of the potential noise image, and when the third sample error is less than the dynamic interval, a set seventh loss function (e.g., 0) may be determined as the second loss function of the potential noise image.

[0100] For example, after identifying the potential noise image set N, it needs to be specially processed differently from other training samples to reduce the impact of noise samples on the model training effect. Existing methods usually discard these samples directly, or modify their category labels based on the model prediction results. The former easily leads to the difficult samples being discarded at the same time, and the latter may artificially introduce new label noise. In order to retain the difficult samples that are misidentified in set N and reduce the impact of noise samples in set N on model training, the present invention adds dynamic intervals to the training samples in set N when calculating the loss function, and determines the interval size according to the frequency with which each sample is identified as a noise sample. This scheme accurately determines the second loss function of the potential noise image based on the comparison of the third sample error and the dynamic interval, and dynamically adjusts the loss function of the suspected noise sample, thereby improving the model's recognition effect on difficult samples and effectively improving the accuracy of image content classification.

[0101] For a potential noise image (x, y) in the potential noise image set N, the intermediate image content classification model predicts the classification result p, and its loss function is h(p, y). Given a third sample error function e(·) that measures the error between the model prediction p and the class label y, the loss function with dynamic interval is The dynamic interval m is determined by the frequency f of the potential noise image (x, y) contained in the potential noise image set N: m = f*∑ max e(p,y). When the frequency f is 0, it means that the potential noise image has never appeared in the potential noise image set N, which makes the interval m 0, h′(p,y) = h(p,y). When the frequency f is 1, it means that the potential noise image is identified as a potential noise image in each round of noise sample recognition, which makes h'(p,y) = 0, which is equivalent to discarding the potential noise image and not participating in training. In other cases, the loss function is calculated for each potential noise image in the noise image set N only when the prediction error e(p,y) is greater than the interval m, thereby affecting the model training. When the prediction error e(p,y) is less than m, it has no impact on the model training, thereby avoiding overfitting of the noise samples.

[0102] S104: Performing a second training on the intermediate image content classification model according to the sample image, the potential noise image, the first loss function, and the second loss function to obtain an image content classification model.

[0103] Exemplarily, based on the above-mentioned multiple sample images, multiple potential noise images, the first loss function, and the second loss function, the intermediate image content classification model is subjected to a second training to obtain an image content classification model. For example, the first loss function is used as the loss function of the sample image, and the second loss function is used as the loss function of the potential noise image, and the intermediate image content classification model is subjected to a second training to obtain an image content classification model. In one embodiment, after completing the second training of the intermediate image content classification model in each round, the obtained image content classification model can be used as the intermediate image content classification model for the second training of the next round, and the training is iterated multiple times until the set training round is reached, or the set training end condition is met (for example, the recognition accuracy reaches the set accuracy), thereby obtaining a trained image content classification model.

[0104] In the above, the image content classification model analyzes and processes the image to be identified to obtain a target label for the image to be identified. The image content classification model is trained based on a set loss function using sample images and potential noise images. The potential noise images are obtained by performing data augmentation and noise sample identification on the sample images. The loss function is determined based on the frequency with which potential noise images are identified as potential noise images. By identifying and retaining potential noise samples and training the image content classification model based on a dynamically determined loss function, the negative impact of noise samples can be effectively reduced during training while retaining difficult samples that are difficult to distinguish, thereby improving the model's recognition performance for difficult samples and effectively improving image content classification accuracy. Furthermore, the noise sample identification scheme based on data augmentation effectively improves the accuracy of noise sample identification, reduces misidentification of clean and difficult samples, and thus improves the final model training results. The model eliminates the need for multiple models or multi-stage processing, effectively reducing the additional computational and time overhead associated with noise processing. Furthermore, a loss function adjustment method based on dynamic intervals can reduce the negative impact of noise samples while retaining difficult samples for training, thereby improving the model's recognition performance for difficult samples.

[0105] FIG6 is a schematic diagram of the structure of an image content classification device provided by an embodiment of the present application. Referring to FIG6 , the image content classification device includes an image acquisition module 61 and an image classification module 62 .

[0106] Among them, the image acquisition module 61 is configured to acquire the image to be identified; the image classification module 62 is configured to input the image to be identified into the trained image content classification model, and analyze and process the image to be identified through the image content classification model to obtain the target label of the image to be identified. The image content classification model is based on the set loss function and is trained according to the sample image and the potential noise image. The potential noise image is obtained by data enhancement of the sample image and noise sample recognition. The loss function is determined according to the frequency with which the potential noise image is identified as a potential noise image.

[0107] In the above, the image to be identified is analyzed and processed by the image content classification model to obtain the target label of the image to be identified, wherein the image content classification model is based on the set loss function and is trained according to the sample image and the potential noise image. The potential noise image is obtained by performing data enhancement on the sample image and identifying the noise sample. The loss function is determined according to the frequency with which the potential noise image is identified as a potential noise image. By identifying and retaining the potential noise samples and training the image content classification model based on the dynamically determined loss function, the negative impact of the noise samples in the training process can be effectively reduced while retaining the difficult samples that are difficult to distinguish to participate in the training, thereby improving the model's recognition effect on difficult samples and effectively improving the accuracy of image content classification.

[0108] In a possible embodiment, the system further includes a model training module configured to train the image content classification model, wherein the model training module is configured to train the image content classification model as follows:

[0109] Performing a first training on the initial image content classification model according to the sample image and the first loss function to obtain an intermediate image content classification model;

[0110] Performing enhancement processing on the sample image to obtain an enhanced image, and performing noise sample recognition on the enhanced image to obtain a potential noise image;

[0111] determining a second loss function for the latent noise image based on the sample image, the latent noise image, and a frequency at which the latent noise image is identified as a latent noise image;

[0112] The intermediate image content classification model is trained for the second time based on the sample image, the potential noise image, the first loss function, and the second loss function to obtain an image content classification model.

[0113] In one possible embodiment, the model training module performs noise sample identification on the enhanced image to obtain a potential noise image, and is configured as follows:

[0114] determining a first noise image from the enhanced image based on consistency scores between the sample image and the enhanced image; and / or

[0115] Based on the prediction error distribution between the sample image and the enhanced image, a second noise image is determined from the enhanced image.

[0116] In one possible embodiment, the model training module determines the first noise image from the enhanced image based on the score consistency between the sample image and the enhanced image, and is configured as follows:

[0117] Determining a model score for each enhanced image using an intermediate image content classification model;

[0118] Determine the noise index of each enhanced image based on the model score and the category label corresponding to the sample image;

[0119] A first noisy image is determined from the enhanced image according to the noise index.

[0120] In one possible embodiment, the model training module determines the noise index of each enhanced image based on the model score and the category label corresponding to the sample image, and is configured as follows:

[0121] Determine the first sample error and sample standard deviation between the model score and the class label corresponding to the sample image;

[0122] The noise index of each enhanced image is determined according to the first sample error and the sample standard deviation.

[0123] In one possible embodiment, the model training module determines the first sample error and the sample standard deviation between the model score and the category label corresponding to the sample image, and is configured as follows:

[0124] Determine, based on a set third loss function, a first sample error between an average model score of each enhanced image and a category label corresponding to the sample image;

[0125] Based on the set fourth loss function, the second sample error between the model score of each enhanced image and the category label corresponding to the sample image is determined, and the sample standard deviation between the model score and the category label corresponding to the sample image is determined according to the standard deviation of the second sample error.

[0126] In one possible embodiment, the model training module determines a first noisy image from the enhanced image according to a noise index, and is configured as follows:

[0127] A first noise image is determined from the enhanced image according to the ranking of the noise indicators and the set first ratio.

[0128] In one possible embodiment, the model training module determines the second noise image from the enhanced image based on the prediction error distribution between the sample image and the enhanced image, and is configured as follows:

[0129] determining a first prediction error distribution for the sample image and a second prediction error distribution for the enhanced image;

[0130] Determining a likelihood ratio that the second sample error corresponding to each enhanced image obeys the first prediction error distribution and the second prediction error distribution;

[0131] A second noise image is determined from the enhanced image according to the likelihood ratio.

[0132] In one possible embodiment, the model training module performs noise sample identification on the enhanced image to obtain a potential noise image, and is configured as follows:

[0133] determining a first noise image from the enhanced image based on a consistency of scores between the sample image and the enhanced image;

[0134] determining a second noise image from the enhanced image based on a prediction error distribution between the sample image and the enhanced image;

[0135] A latent noise image is determined according to the intersection of each of the first noise image and the second noise image.

[0136] In one possible embodiment, the model training module determines a second loss function for the potential noise image based on the sample image, the potential noise image, and the frequency with which the potential noise image is identified as a potential noise image, and is configured as follows:

[0137] Determining a third sample error between the model score of each potential noise image and the category label corresponding to the sample image based on the set fifth loss function;

[0138] A second loss function for the potential noise image is determined based on the third sample error and a frequency at which the potential noise image is identified as a potential noise image.

[0139] In one possible embodiment, the model training module determines a second loss function for the potential noise image based on the third sample error and the frequency at which the potential noise image is identified as a potential noise image, and is configured as follows:

[0140] determining a dynamic interval of each potential noise image based on the third sample error and a frequency at which the potential noise image is identified as a potential noise image;

[0141] A second loss function for the potential noisy image is determined based on the dynamic interval and the third sample error.

[0142] In one possible embodiment, the model training module determines a second loss function of the potential noise image according to the dynamic interval and the third sample error, and is configured as follows:

[0143] When the third sample error is greater than or equal to the dynamic interval, determining the set sixth loss function as the second loss function of the potential noise image;

[0144] When the third sample error is less than the dynamic interval, the set seventh loss function is determined as the second loss function of the potential noise image.

[0145] It is worth noting that in the embodiment of the above-mentioned image content classification device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the embodiments of this application.

[0146] The embodiment of the present application also provides an image content classification device, which can integrate the image content classification apparatus provided in the embodiment of the present application. Figure 7 is a structural diagram of an image content classification device provided in the embodiment of the present application. Referring to Figure 7, the image content classification device includes: an input device 73, an output device 74, a memory 72, and one or more processors 71; the memory 72 is used to store one or more programs; when the one or more programs are executed by one or more processors 71, the one or more processors 71 implement the image content classification method provided in the above embodiment. The above-mentioned image content classification apparatus, device, and computer can be used to execute the image content classification method provided in any of the above embodiments, and have corresponding functions and beneficial effects.

[0147] An embodiment of the present application also provides a non-volatile storage medium that stores computer-executable instructions, which, when executed by a computer processor, are used to execute the image content classification method provided in the above embodiment. Of course, the non-volatile storage medium that stores computer-executable instructions provided in an embodiment of the present application, whose computer-executable instructions are not limited to the image content classification method provided above, can also execute related operations in the image content classification method provided in any embodiment of the present application. The image content classification apparatus, device, and storage medium provided in the above embodiments can execute the image content classification method provided in any embodiment of the present application. For technical details not fully described in the above embodiments, please refer to the image content classification method provided in any embodiment of the present application.

[0148] Based on the above embodiments, the embodiments of the present application also provide a computer program product. The essence of the technical solution of the present application or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes a number of instructions for enabling a computer device, a mobile terminal or the processor therein to execute all or part of the steps of the image content classification method provided in each embodiment of the present application.

Claims

1. A method for classifying image content, wherein: include: Obtain the image to be recognized; The image to be identified is input into a trained image content classification model, and the image to be identified is analyzed and processed by the image content classification model to obtain a target label of the image to be identified. The image content classification model is trained based on a set loss function according to sample images and potential noise images. The potential noise images are obtained by performing data enhancement and noise sample identification on the sample images. The loss function is determined according to the frequency with which the potential noise image is identified as a potential noise image.

2. The image content classification method according to claim 1, wherein: The training process of the image content classification model includes: Performing a first training on the initial image content classification model according to the sample image and the first loss function to obtain an intermediate image content classification model; Performing enhancement processing on the sample image to obtain an enhanced image, and performing noise sample identification on the enhanced image to obtain a potential noise image; determining a second loss function for the potential noise image according to the sample image, the potential noise image, and a frequency at which the potential noise image is identified as a potential noise image; The intermediate image content classification model is subjected to a second training according to the sample image, the potential noise image, the first loss function, and the second loss function to obtain an image content classification model.

3. The image content classification method according to claim 2, wherein: The step of performing noise sample identification on the enhanced image to obtain a potential noise image includes: determining a first noise image from the enhanced image based on the consistency of scores between the sample image and the enhanced image; and / or Based on the prediction error distribution between the sample image and the enhanced image, a second noise image is determined from the enhanced image.

4. The image content classification method according to claim 3, wherein: The determining a first noise image from the enhanced image based on the score consistency between the sample image and the enhanced image includes: determining a model score for each of the enhanced images using the intermediate image content classification model; Determining a noise index of each of the enhanced images according to the model score and the category label corresponding to the sample image; A first noise image is determined from the enhanced image according to the noise index.

5. The image content classification method according to claim 4, wherein: Determining the noise index of each enhanced image according to the model score and the category label corresponding to the sample image includes: Determining a first sample error and a sample standard deviation between the model score and the category label corresponding to the sample image; The noise index of each enhanced image is determined according to the first sample error and the sample standard deviation.

6. The image content classification method according to claim 5, wherein: Determining a first sample error and a sample standard deviation between the model score and the category label corresponding to the sample image includes: Determining, based on a set third loss function, a first sample error between an average model score of each of the enhanced images and a category label corresponding to the sample image; Based on the set fourth loss function, the second sample error between the model score of each enhanced image and the category label corresponding to the sample image is determined, and the sample standard deviation between the model score and the category label corresponding to the sample image is determined according to the standard deviation of the second sample error.

7. The image content classification method according to claim 4, wherein: Determining a first noise image from the enhanced image according to the noise index includes: A first noise image is determined from the enhanced image according to the ranking of the noise indicators and the set first ratio.

8. The image content classification method according to claim 3, wherein: The determining of a second noise image from the enhanced image based on a prediction error distribution between the sample image and the enhanced image comprises: determining a first prediction error distribution of the sample image and a second prediction error distribution of the enhanced image; Determining a likelihood ratio that a second sample error corresponding to each of the enhanced images obeys the first prediction error distribution and the second prediction error distribution; A second noise image is determined from the enhanced image according to the likelihood ratio.

9. The image content classification method according to claim 2, wherein: The step of performing noise sample identification on the enhanced image to obtain a potential noise image includes: determining a first noise image from the enhanced image based on a score consistency between the sample image and the enhanced image; determining a second noise image from the enhanced image based on a prediction error distribution between the sample image and the enhanced image; A potential noise image is determined according to the intersection of the first noise images and the second noise image.

10. The image content classification method according to claim 1, wherein: The determining, according to the sample image, the potential noise image, and the frequency at which the potential noise image is identified as a potential noise image, a second loss function of the potential noise image includes: Determining, based on a set fifth loss function, a third sample error between the model score of each of the potential noise images and the category label corresponding to the sample image; A second loss function for the potential noise image is determined according to the third sample error and a frequency at which the potential noise image is identified as a potential noise image.

11. The image content classification method according to claim 10, wherein: The determining, based on the third sample error and the frequency at which the potential noise image is identified as a potential noise image, a second loss function of the potential noise image comprises: determining a dynamic interval of each of the potential noise images according to the third sample error and a frequency at which the potential noise image is identified as a potential noise image; A second loss function for the potential noisy image is determined based on the dynamic interval and the third sample error.

12. The image content classification method according to claim 11, wherein: The determining a second loss function of the potential noise image according to the dynamic interval and the third sample error includes: In a case where the third sample error is greater than or equal to the dynamic interval, determining the set sixth loss function as the second loss function of the potential noise image; In a case where the third sample error is smaller than the dynamic interval, the set seventh loss function is determined as the second loss function of the potential noise image.

13. An image content classification device, wherein: It includes an image acquisition module and an image classification module, wherein: The image acquisition module is configured to acquire an image to be identified; The image classification module is configured to input the image to be identified into a trained image content classification model, analyze and process the image to be identified through the image content classification model, and obtain a target label for the image to be identified. The image content classification model is trained based on a set loss function according to sample images and potential noise images. The potential noise images are obtained by performing data enhancement and noise sample identification on the sample images. The loss function is determined based on the frequency with which the potential noise image is identified as a potential noise image.

14. An image content classification device, wherein: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the image content classification method according to any one of claims 1 to 12.

15. A non-volatile storage medium storing computer-executable instructions, wherein: When the computer executable instructions are executed by a computer processor, they are used to perform the image content classification method according to any one of claims 1 to 12.

16. A computer program product comprising a computer program, wherein When the computer program is executed by a processor, the image content classification method according to any one of claims 1 to 12 is implemented.

Images