Image recognition method and device based on noisy data
Through multi-model interactive algorithm screening and noise recognition model optimization, the identification model accuracy problem caused by inaccurate sample type recognition is solved, the accuracy and stability of the image recognition model is improved, and the problems of labeling errors and noise data are solved.
Patent Information
- Application Number
- CN202411098781.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-08-12
AI Technical Summary
In the prior art, the sample type identification and labeling are not accurate enough, resulting in poor accuracy of the trained recognition model, especially in the fields of medical images, due to the influence of labeling errors and noise data, the recognition model performance is degraded, making it difficult to accurately identify the image. object.
Using deep learning-based image recognition method, noise data and difficult-to-classify samples are screened out through multi-model interaction algorithms, clean samples and noise sample data sets are constructed, pre-trained classification recognition model is used for image recognition, and further use noise recognition model to perform further Filter, optimize the model training process, and improve identification accuracy.
By accurately identifying the sample type, the accuracy of the trained recognition model is improved, the interference of noise data on model training is overcome, the accuracy and stability of the recognition results are enhanced, and the model is more valuable in practical applications.
Smart Images

Figure CN118747829B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image recognition and data processing, and in particular to an image recognition method and device based on noisy data. Background Art
[0002] In the existing technology, artificial intelligence recognition systems are often used when performing image recognition such as medical images. When building an artificial intelligence recognition system, it is necessary to collect a large amount of target data, and the collected data must be accurately labeled by field experts or relevant labeling personnel. However, for the same labeled data, different labelers may give different labeling results. For example, in a natural data set, if an image contains different objects, such as cats, tigers, lions, etc., due to subjective factors, labelers may miss or mislabel the data, resulting in incorrectly labeled information in the labeled data set. Especially for medical data, due to special factors such as the professionalism of the field, different labelers for the same batch of data may have different labeling results, resulting in ambiguity in the labeled database, which seriously affects the purity and reliability of the labeled data, causing the performance of the recognition model to drop significantly, outputting incorrect recognition results, and limiting the practical application of the recognition model. Moreover, in the process of training the recognition model, it is usually assumed that the training data set is labeled completely correctly without any labeling errors, and the situation where the training database contains incorrect labeling is not considered. At the same time, the methods used in the prior art usually treat difficult-to-identify samples as noise samples. Incorrectly labeled data and high-value difficult-to-identify samples mistakenly treated as noise samples will affect the accuracy of the trained recognition model.
[0003] In view of this, the present invention provides an image recognition method and device based on noisy data to solve the problem in the prior art that the trained recognition model has poor accuracy due to inaccurate sample type recognition and labeling. Summary of the invention
[0004] The present invention provides an image recognition method and device based on noisy data to solve the problem in the prior art that the accuracy of a trained recognition model is poor due to inaccurate sample type recognition and labeling, and improves the accuracy of the trained recognition model by accurately identifying samples.
[0005] The present invention provides an image recognition method based on noisy data, the method comprising:
[0006] Obtaining an original image to be recognized;
[0007] Inputting the original image to be recognized into a pre-trained classification recognition model to obtain an image recognition result output by the classification recognition model;
[0008] The classification and recognition model is based on a deep learning model and is trained using a pre-constructed image sample data set, wherein the image sample data set includes a clean sample data set and a noise sample data set;
[0009] The clean sample data set includes a clean sample set obtained by screening out a difficult-to-classify sample set from the original data sample, and a difficult-to-identify sample set obtained by screening the difficult-to-classify sample set through a noise recognition model;
[0010] The noise sample data set is composed of a noise sample set obtained after the difficult-to-classify sample set is screened by the noise recognition model, and the difficult-to-classify sample set is obtained by iteratively screening the original data sample set through a multi-model interaction algorithm.
[0011] In some embodiments, the original data samples are iteratively screened by a multi-model interaction algorithm to obtain the difficult-to-classify samples, specifically including:
[0012] Inputting the target sample into a first classification model to obtain a first probability output by the first classification model that the target sample belongs to a target category label;
[0013] When the first probability is less than a first probability threshold, inputting the target sample into a second classification model to obtain a second probability output by the second classification model that the target sample belongs to the target class label;
[0014] When the second probability is less than a second probability threshold, determining that the target sample is a difficult-to-classify sample;
[0015] Each of the original data samples in the original data sample set is sequentially used as a target sample for iteration to obtain all difficult-to-classify samples in the original data sample set.
[0016] In some embodiments, the target sample is input into a first classification model to obtain a first probability output by the first classification model that the target sample belongs to a target category label, and then the method further includes:
[0017] When the first probability is greater than a first probability threshold, the target sample is classified into the clean sample set.
[0018] In some embodiments, obtaining a second probability that the target sample output by the second classification model belongs to the target class label further includes:
[0019] When the second probability is greater than a second probability threshold, the target sample is classified into the clean sample set.
[0020] In some embodiments, after the difficult-to-classify sample set is screened by the noise recognition model, a noise sample set is obtained, which specifically includes:
[0021] Input each sample in the difficult-to-classify sample set into the first sub-model and the second sub-model of the noise recognition model respectively, so as to obtain a first sub-probability output by the first sub-model and a second sub-probability output by the second sub-model;
[0022] Calculating weighted values of the first sub-probability and the second sub-probability according to the contribution;
[0023] When the weighted value is greater than the third probability, the sample is determined to be a noise sample.
[0024] In some embodiments, the first sub-model is obtained by training using a sample set and a class label within a preset interval of class probability; the sample set within the preset interval is obtained by inputting a clean sample set and a simulated noise sample set extracted from the clean sample set into a simulated recognition model;
[0025] The second sub-model is obtained by using the predicted probability of each sample in each round of iteration as input and performing optimization training with expectation maximization as the optimization objective.
[0026] The present invention also provides an image recognition device based on noisy data, the device comprising:
[0027] An image acquisition unit, used for acquiring an original image to be recognized;
[0028] A result output unit, used for inputting the original image to be identified into a pre-trained classification and recognition model to obtain an image recognition result output by the classification and recognition model;
[0029] The classification and recognition model is based on a deep learning model and is trained using a pre-constructed image sample data set, wherein the image sample data set includes a clean sample data set and a noise sample data set;
[0030] The clean sample data set includes a clean sample set obtained by screening out a difficult-to-classify sample set from the original data sample, and a difficult-to-identify sample set obtained by screening the difficult-to-classify sample set through a noise recognition model;
[0031] The noise sample data set is composed of a noise sample set obtained after the difficult-to-classify sample set is screened by the noise recognition model, and the difficult-to-classify sample set is obtained by iteratively screening the original data sample set through a multi-model interaction algorithm.
[0032] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described above when executing the program.
[0033] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method described above when executed by a processor.
[0034] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method described above is implemented.
[0035] The image recognition method based on noisy data provided by the present invention obtains the original image to be recognized, and inputs the original image to be recognized into a pre-trained classification recognition model to obtain the image recognition result output by the classification recognition model; wherein the classification recognition model is based on a deep learning model, and is trained using a pre-constructed image sample data set, and the image sample data set includes a clean sample data set and a noise sample data set; the clean sample data set includes a clean sample set obtained by screening out a difficult-to-classify sample set from the original data sample, and a difficult-to-recognize sample set obtained by screening the difficult-to-classify sample set through a noise recognition model; the noise sample data set is composed of a noise sample set obtained by screening the difficult-to-classify sample set through the noise recognition model, and the difficult-to-classify sample set is obtained by iteratively screening the original data sample set through a multi-model interaction algorithm.
[0036] In this way, the method and device provided by the present invention utilize the interaction of multiple models to screen out possible noise data in the data, and overcome the interference of difficult sample data on noise data mining, so that the difficult sample data can better optimize the model; at the same time, the multi-model interaction is utilized to jointly distinguish difficult sample (i.e., difficult to identify sample) data and noise data, so that the discrimination result is more accurate; when optimizing the model, the mined noise data is fully utilized, all data is fully and efficiently utilized for model optimization, and the model output result is calibrated. The problem of poor accuracy of the trained recognition model due to inaccurate sample type identification and labeling in the prior art is solved, and the accuracy of the trained recognition model is improved by accurately identifying the sample. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0038] Figure 1 One of the flow charts of the image recognition method based on noisy data provided by the present invention;
[0039] Figure 2 The second flowchart of the image recognition method based on noisy data provided by the present invention;
[0040] Figure 3 This is the model disturbance effect diagram;
[0041] Figure 4 The third flowchart of the image recognition method based on noisy data provided by the present invention;
[0042] Figure 5 The fourth flowchart of the image recognition method based on noisy data provided by the present invention;
[0043] Figure 6 This is a schematic diagram of the distribution of noisy data and clean data before data processing;
[0044] Figure 7 Flow chart 5 of the image recognition method based on noisy data provided by the present invention;
[0045] Figure 8 This is a distribution diagram of noise data and difficult sample data after data processing;
[0046] Fig. 9 A schematic diagram of the structure of an image recognition device based on noisy data provided by the present invention;
[0047] Fig.10 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0048] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0049] Please refer to Figure 1 , Figure 1One of the flow charts of the image recognition method based on noisy data provided by the present invention;
[0050] In a specific embodiment, the image recognition method based on noisy data provided by the present invention comprises the following steps:
[0051] S110: Acquire the original image to be identified. It should be understood that when collecting data, different types of data are collected in different ways; for example, for medical data, images can be acquired with the help of electron microscopes, phase contrast microscopes, and optical microscopes, and data can be collected using microscope cameras; while for natural data, data can be collected using ordinary cameras or networks.
[0052] During the model training process, the data needs to be labeled, such as selecting the target area on the original image. When labeling data, taking medical data as an example, professional doctors can use specific labeling tools to label information such as categories. For natural data, labelers in related fields can use labeling tools such as Label Me to label the data. When the labeled amount reaches a certain scale, the labeled data will be divided into training set, validation set and test set according to a certain ratio to prepare data for building the model.
[0053] S120: Inputting the original image to be recognized into a pre-trained classification recognition model to obtain an image recognition result output by the classification recognition model;
[0054] The classification and recognition model is based on a deep learning model and is trained using a pre-constructed image sample data set, wherein the image sample data set includes a clean sample data set and a noise sample data set;
[0055] The clean sample data set includes a clean sample set obtained by screening out a difficult-to-classify sample set from the original data sample, and a difficult-to-identify sample set obtained by screening the difficult-to-classify sample set through a noise recognition model;
[0056] The noise sample data set is composed of a noise sample set obtained after the difficult-to-classify sample set is screened by the noise recognition model, and the difficult-to-classify sample set is obtained by iteratively screening the original data sample set through a multi-model interaction algorithm.
[0057] Specifically, when constructing an image sample dataset, first, a multi-model interaction algorithm is used to qualitatively determine whether the original sample is a noise sample, and possible noise samples in the training data are found; for the possible noise samples (i.e., difficult-to-classify samples) found, the recognition model is further used to identify the difficult samples (i.e., difficult-to-identify samples) and noise data (i.e., noise samples) in the difficult-to-classify samples, so that the easily confused "difficult samples" and "noise data" are separated, and the noise data is better utilized; finally, the screened noise sample dataset is combined with the remaining clean sample dataset and sent to the deep learning network for model training to obtain a classification and recognition model.
[0058] In a specific usage scenario, such as Figure 2 As shown, the present invention constructs an intelligent recognition model based on multi-model interaction. Generally speaking, a training data set is first constructed, and after the data set is labeled, it is divided into a training set and a test set; the training set and the test set are used to train the model respectively, and the trained model is packaged to form an intelligent recognition model; during use, a database is constructed after data collection, the intelligent recognition model is called, the sample category is obtained, the number of each category is counted, and the statistical results are output.
[0059] Specifically, first, the multi-model interaction method is used to iterate step by step to find samples that are difficult to classify correctly (i.e., a set of difficult-to-classify samples, including difficult samples and noise samples). The remaining samples are clean samples, and a recognition model that distinguishes difficult samples from noise samples is learned based on the clean samples. Then, the recognition model is used to divide the difficult-to-classify sample set to obtain a difficult-to-identify sample set and a noise sample set. Finally, based on the noise sample set and clean sample data set obtained by the division (including the clean sample set and the difficult-to-identify sample set), the final intelligent recognition model is trained to obtain the target model.
[0060] Due to the presence of noise samples in the data set, it is impossible to use traditional artificial intelligence methods to include noise samples in the training set for model learning. In order to fully utilize noise samples and clean samples, this embodiment proposes a loss function based on the prediction consistency principle, that is, for a certain input data, the data is perturbed and input into the model after the perturbation. The result of the model output should be the same as the result of the model output before the perturbation, which is the consistency principle. The output result of the model before the perturbation is used as the standard result of the output result of the model after the perturbation, and the result of the model output after the perturbation is optimized to be close to the standard result. The specific optimization goals are as follows:
[0061] ;
[0062] in, is the output of the model before disturbance, is the output result of the model after disturbance. In this embodiment, it is hoped that the two are as close as possible, such as Figure 3 As shown, the upper left image is the image before perturbation, and the other images are the images after perturbation.
[0063] like Figure 4 As shown in the figure, in order to improve the model effect, after obtaining the classification recognition model, the noise data in the noise sample set is used to correct the model output results. For the noise data, the model predicts its possible label. However, the model prediction results themselves are uncertain, and the uncertainty of the model prediction results needs to be calibrated. Before calibration, the noise transfer matrix is calculated using the clean data that has been screened. This matrix reflects the probability that a certain category is misclassified as other categories, that is, the elements in the matrix P It represents the probability that the model identifies the i-th class sample as the j-th class sample.
[0064] After obtaining the noise transfer matrix, the prediction results output by the model can be calibrated. The calibration method is as follows:
[0065] ;
[0066] Among them, Y is the real label of the sample, P is the result predicted by the model, and T is the noise transfer matrix. According to the above method, after obtaining the prediction result of the model, the result is calibrated to its real label, and the real label is used as the standard label of the noise sample for optimization learning under the consistency principle.
[0067] Finally, the noise samples and difficult-to-identify samples were completely divided into two data sets, the model output results were calibrated, and the consistency principle was used to effectively utilize both the noise samples and the clean samples, which increased the data utilization efficiency, improved the recognition performance and stability of the model, and made the model have extremely high practical application value.
[0068] After the model training is completed, the image to be classified is input into the model to obtain the specific category of the image and the specific number of each type of image in the input set, and the final statistical result can be obtained.
[0069] In some embodiments, Figure 5 As shown, the original data samples are iteratively screened by a multi-model interaction algorithm to obtain the difficult-to-classify samples, which specifically includes the following steps:
[0070] S510: Inputting the target sample into a first classification model to obtain a first probability output by the first classification model that the target sample belongs to a target category label;
[0071] S520: When the first probability is less than the first probability threshold, input the target sample into the second classification model to obtain the second probability that the target sample belongs to the target class label output by the second classification model; otherwise, when the first probability is greater than the first probability threshold, classify the target sample into the clean sample set;
[0072] S530: When the second probability is less than the second probability threshold, determine that the target sample is a difficult-to-classify sample; otherwise, when the second probability is greater than the second probability threshold, classify the target sample into the clean sample set;
[0073] S540: Iteratively use each of the original data samples in the original data sample set as the target sample to obtain all the difficult-to-classify samples in the original data sample set.
[0074] Still taking Figure 4 the specific usage scenario shown as an example, in the process of multi-model interaction to identify difficult-to-classify samples to form a set of difficult-to-classify samples, in order to identify noise samples, in this embodiment, two models with different structures and different numbers of parameters (i.e., the first classification model and the second classification model) are used to judge whether the same sample is a noise sample.
[0075] As Figure 6 shown, some noise data is mixed in the clean data, which has a serious impact on the determination of the classification boundary. In order to find out the noise data confused in the above figure and make it easier for the model to determine the classification boundary, first, the original data is preprocessed and then input into the first classification model (i.e., model A). Model A outputs a probability distribution, which represents the judgment of model A on the sample; if the probability p1 of the true class label corresponding to the sample is greater than the threshold t1, then the sample will not be considered a difficult-to-classify sample; if p1 < t1, then the sample is input into model B for further judgment. If the probability p2 of the sample output by the second classification model (i.e., model B) is greater than the threshold t2, then the sample will not be considered a difficult-to-classify sample, that is, the sample is a clean sample; otherwise, the sample is considered a difficult-to-classify sample; only when both models consider a sample to be a difficult-to-classify sample, will the sample finally be classified into the set of difficult-to-classify samples. The above process is iterated N rounds, and the sample set of the difficult-to-classify samples in the last round is selected as the final data set; at the same time, record the prediction probability p of each sample in each round of iteration.
[0076] In some embodiments, as Figure 7 shown, after the set of difficult-to-classify samples is screened by the noise recognition model, a set of noise samples is obtained, which specifically includes the following steps:
[0077] S710: Input each sample in the difficult-to-classify sample set into the first sub-model and the second sub-model of the noise recognition model respectively, so as to obtain a first sub-probability output by the first sub-model and a second sub-probability output by the second sub-model;
[0078] S720: Calculate weighted values of the first sub-probability and the second sub-probability according to the contribution;
[0079] S730: When the weighted value is greater than the third probability, determine that the sample is a noise sample.
[0080] When performing noise sample recognition, please continue to refer to Figure 4 This embodiment adopts a joint judgment method of implicit probability model and small discriminant model to improve the data processing effect.
[0081] For a certain sample, the hidden probability model and the small discriminant model give the probability of it being a difficult-to-identify sample and a noise sample, respectively, marked as p1 and p2; based on the prediction results given respectively, the discriminant results given by the two models are considered; the grid search technology is used to determine the contribution of the two models to the final result; assuming that the contribution of the first model to the final result is m, then the contribution of the second model to the final result is 1-m, at this time, the final judgment result of the sample is given by the following formula:
[0082] ;
[0083] in, is the probability value of the sample being predicted as a noise sample, m is a hyperparameter, and the optimal solution is obtained by grid search technology; if a sample , then the sample is classified as a noise sample set, otherwise the sample is classified as a clean sample set.
[0084] Based on the above method, noise samples and difficult samples can be distinguished, and a complete clean sample data set and a noise sample data set can be divided.
[0085] Among them, the first sub-model is trained using a sample set with class probabilities within a preset interval and class labels; the sample set within the preset interval is obtained by inputting a clean sample set and a simulated noise sample set extracted from the clean sample set into a simulated recognition model. When constructing a small discriminative model to distinguish hard samples and noise samples, for a set of hard-to-classify samples, it is necessary to distinguish which are correctly labeled hard samples and which are mislabeled noise samples in the set; samples in the original data that are not included in the hard-to-classify sample set are clean samples. Select 20% of the clean sample set, randomly shuffle its labeling results to obtain a virtual noise sample set, and use this virtual noise sample set and the hard-to-classify sample set to construct a lightweight recognition model to distinguish hard samples and noise samples.
[0086] Specifically, the clean sample set and the artificially created noise sample set are grouped into the same training data set, and this data set is input into a recognition model, which outputs the probability p that the input sample belongs to a certain class; select samples where t1 < p < t2, that is, a set of samples that are difficult to identify in an intermediate state. Since the true labeling results of all samples in this set are known, it is possible to know which samples are hard samples and which samples are noise samples; form a set with the selected hard samples and noise samples, and use this set to train a lightweight recognition model to distinguish whether a sample is a hard sample or a noise sample. As Figure 8 shown, by using this model to define the classification boundary for noise data and hard sample data, the small discriminative model can accurately and efficiently divide and determine hard sample data and noise data.
[0087] In step S710, the second sub-model is optimized and trained using the prediction probability of each sample in each round of iteration as the input and the expectation maximization as the optimization objective. Please continue to refer to Figure 4 , the second sub-model is a hidden probability model, and a hidden probability model is constructed to assist in the identification of hard samples and noise samples. To better identify hard samples and noise samples, in this embodiment, a latent variable model is built to assist in the identification of the above tasks. The latent variable represents the probability distribution of a sample belonging to two classes. For the prediction probability of each sample in each round of iteration retained in the previous steps, this prediction probability is used as the input of the latent variable model to predict the latent variable distribution, and finally the probability that each sample belongs to each class is obtained. Among them, this embodiment uses the expectation maximization algorithm to predict the latent variable distribution to obtain the specific probability that a sample belongs to a certain class; the optimization objective of expectation maximization is as follows:
[0088] ;
[0089] Among them, Q(z) is the posterior distribution of the latent variable z, L(θ) is the expectation of the joint distribution P(x, z) under the distribution Q(z), that is, the likelihood of the joint distribution P(x, z); we hope to maximize the expectation / likelihood, estimate the parameter θ, and at the same time estimate the posterior distribution of z under the parameter θ and sample x, that is, the probability distribution of the sample belonging to a certain category. In this way, we can get the possibility that a sample that is difficult to identify belongs to a noise sample.
[0090] In the above-mentioned specific implementation manner, the image recognition method based on noisy data provided by the present invention obtains the original image to be recognized, and inputs the original image to be recognized into a pre-trained classification and recognition model to obtain the image recognition result output by the classification and recognition model; wherein the classification and recognition model is based on a deep learning model, and is trained using a pre-constructed image sample data set, and the image sample data set includes a clean sample data set and a noise sample data set; the clean sample data set includes a clean sample set obtained after screening out a difficult-to-classify sample set from the original data sample, and a difficult-to-recognize sample set obtained after the difficult-to-classify sample set is screened by the noise recognition model; the noise sample data set is composed of a noise sample set obtained after the difficult-to-classify sample set is screened by the noise recognition model, and the difficult-to-classify sample set is obtained by iteratively screening the original data sample set through a multi-model interaction algorithm.
[0091] In this way, the method and device provided by the present invention utilize the interaction of multiple models to screen out possible noise data in the data, and overcome the interference of difficult sample data on noise data mining, so that the difficult sample data can better optimize the model; at the same time, the multi-model interaction is utilized to jointly distinguish difficult sample (i.e., difficult to identify sample) data and noise data, so that the discrimination result is more accurate; when optimizing the model, the mined noise data is fully utilized, all data is fully and efficiently utilized for model optimization, and the model output result is calibrated. The problem of poor accuracy of the trained recognition model due to inaccurate sample type identification and labeling in the prior art is solved, and the accuracy of the trained recognition model is improved by accurately identifying the sample.
[0092] In addition to the above method, the present invention also provides an image recognition device based on noisy data, such as Fig. 9 As shown, the device comprises:
[0093] An image acquisition unit 910 is used to acquire an original image to be recognized;
[0094] A result output unit 920 is used to input the original image to be recognized into a pre-trained classification recognition model to obtain an image recognition result output by the classification recognition model;
[0095] The classification and recognition model is based on a deep learning model and is trained using a pre-constructed image sample data set, wherein the image sample data set includes a clean sample data set and a noise sample data set;
[0096] The clean sample data set includes a clean sample set obtained by screening out a difficult-to-classify sample set from the original data sample, and a difficult-to-identify sample set obtained by screening the difficult-to-classify sample set through a noise recognition model;
[0097] The noise sample data set is composed of a noise sample set obtained after the difficult-to-classify sample set is screened by the noise recognition model, and the difficult-to-classify sample set is obtained by iteratively screening the original data sample set through a multi-model interaction algorithm.
[0098] In some embodiments, the original data samples are iteratively screened by a multi-model interaction algorithm to obtain the difficult-to-classify samples, specifically including:
[0099] Inputting the target sample into a first classification model to obtain a first probability output by the first classification model that the target sample belongs to a target category label;
[0100] When the first probability is less than a first probability threshold, inputting the target sample into a second classification model to obtain a second probability output by the second classification model that the target sample belongs to the target class label;
[0101] When the second probability is less than a second probability threshold, determining that the target sample is a difficult-to-classify sample;
[0102] Each of the original data samples in the original data sample set is sequentially used as a target sample for iteration to obtain all difficult-to-classify samples in the original data sample set.
[0103] In some embodiments, the target sample is input into a first classification model to obtain a first probability output by the first classification model that the target sample belongs to a target category label, and then the method further includes:
[0104] When the first probability is greater than a first probability threshold, the target sample is classified into the clean sample set.
[0105] In some embodiments, obtaining a second probability that the target sample output by the second classification model belongs to the target class label further includes:
[0106] When the second probability is greater than a second probability threshold, the target sample is classified into the clean sample set.
[0107] In some embodiments, after the difficult-to-classify sample set is screened by the noise recognition model, a noise sample set is obtained, which specifically includes:
[0108] Input each sample in the difficult-to-classify sample set into the first sub-model and the second sub-model of the noise recognition model respectively, so as to obtain a first sub-probability output by the first sub-model and a second sub-probability output by the second sub-model;
[0109] Calculating weighted values of the first sub-probability and the second sub-probability according to the contribution;
[0110] When the weighted value is greater than the third probability, the sample is determined to be a noise sample.
[0111] In some embodiments, the first sub-model is obtained by training using a sample set and a class label within a preset interval of class probability; the sample set within the preset interval is obtained by inputting a clean sample set and a simulated noise sample set extracted from the clean sample set into a simulated recognition model;
[0112] The second sub-model is obtained by using the predicted probability of each sample in each round of iteration as input and performing optimization training with expectation maximization as the optimization objective.
[0113] In the above-mentioned specific embodiment, the image recognition device based on noisy data provided by the present invention can obtain the image recognition result output by the classification recognition model by acquiring the original image to be recognized and inputting the original image to be recognized into a pre-trained classification recognition model; wherein the classification recognition model is based on a deep learning model and is trained using a pre-constructed image sample data set, and the image sample data set includes a clean sample data set and a noise sample data set; the clean sample data set includes a clean sample set obtained by screening out a difficult-to-classify sample set from the original data sample, and a difficult-to-recognize sample set obtained after the difficult-to-classify sample set is screened by the noise recognition model; the noise sample data set is composed of a noise sample set obtained after the difficult-to-classify sample set is screened by the noise recognition model, and the difficult-to-classify sample set is obtained by iteratively screening the original data sample set through a multi-model interaction algorithm.
[0114] In this way, the device provided by the present invention utilizes the interaction of multiple models to screen out possible noise data in the data, and overcomes the interference of difficult sample data on noise data mining, so that the difficult sample data can better optimize the model; at the same time, the multi-model interaction is used to jointly distinguish difficult sample (i.e., difficult to identify sample) data and noise data, so that the discrimination result is more accurate; when optimizing the model, the mined noise data is fully utilized, all data is fully and efficiently used for model optimization, and the model output result is calibrated. The problem of poor accuracy of the trained recognition model due to inaccurate sample type identification and labeling in the prior art is solved, and the accuracy of the trained recognition model is improved by accurately identifying the sample.
[0115] Fig.10 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.10 As shown, the electronic device may include: a processor 1010, a communication interface 1020, a memory 1030 and a communication bus 1040, wherein the processor 1010, the communication interface 1020 and the memory 1030 communicate with each other through the communication bus 1040. The processor 1010 may call the logic instructions in the memory 1030 to execute the above method.
[0116] In addition, the logic instructions in the above-mentioned memory 1030 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the part of the technical solution of the present invention that essentially contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instruction sets that enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0117] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the above method.
[0118] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0119] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. An image recognition method based on noisy data, characterized in that: The method comprises: Obtaining an original image to be recognized; Inputting the original image to be recognized into a pre-trained classification recognition model to obtain an image recognition result output by the classification recognition model; The classification and recognition model is based on a deep learning model and is trained using a pre-constructed image sample data set, wherein the image sample data set includes a clean sample data set and a noise sample data set; The clean sample data set includes a clean sample set obtained by screening out a difficult-to-classify sample set from the original data sample, and a difficult-to-identify sample set obtained by screening the difficult-to-classify sample set through a noise recognition model; The noise sample data set is composed of a noise sample set obtained after the difficult-to-classify sample set is screened by the noise recognition model, and the difficult-to-classify sample set is obtained by iteratively screening the original data sample set through a multi-model interaction algorithm; The original data samples are iteratively screened by a multi-model interaction algorithm to obtain the difficult-to-classify samples, which specifically includes: Inputting the target sample into a first classification model to obtain a first probability output by the first classification model that the target sample belongs to a target category label; When the first probability is less than a first probability threshold, inputting the target sample into a second classification model to obtain a second probability output by the second classification model that the target sample belongs to the target class label; When the second probability is less than a second probability threshold, determining that the target sample is a difficult-to-classify sample; Iterate each of the original data samples in the original data sample set as a target sample in turn to obtain all difficult-to-classify samples in the original data sample set; The difficult-to-classify sample set is screened by the noise recognition model to obtain a noise sample set, which specifically includes: Input each sample in the difficult-to-classify sample set into the first sub-model and the second sub-model of the noise recognition model respectively, so as to obtain a first sub-probability output by the first sub-model and a second sub-probability output by the second sub-model; Calculating weighted values of the first sub-probability and the second sub-probability according to the contribution; When the weighted value is greater than the third probability, the sample is determined to be a noise sample.
2. The image recognition method based on noisy data according to claim 1, characterized in that: The target sample is input into a first classification model to obtain a first probability output by the first classification model that the target sample belongs to a target category label, and then further includes: When the first probability is greater than a first probability threshold, the target sample is classified into the clean sample set.
3. The image recognition method based on noisy data according to claim 1, characterized in that: Obtaining a second probability that the target sample output by the second classification model belongs to the target class label, and then further comprising: When the second probability is greater than a second probability threshold, the target sample is classified into the clean sample set.
4. The image recognition method based on noisy data according to claim 1, characterized in that: The first sub-model is obtained by training using a sample set and a class label within a preset interval of class probability; the sample set within the preset interval is obtained by inputting a clean sample set and a simulated noise sample set extracted from the clean sample set into a simulated recognition model; The second sub-model is obtained by using the predicted probability of each sample in each round of iteration as input and performing optimization training with expectation maximization as the optimization objective.
5. An image recognition device based on noisy data, used to implement the method according to any one of claims 1 to 4, characterized in that: The device comprises: An image acquisition unit, used for acquiring an original image to be recognized; A result output unit, used for inputting the original image to be identified into a pre-trained classification and recognition model to obtain an image recognition result output by the classification and recognition model; The classification and recognition model is based on a deep learning model and is trained using a pre-constructed image sample data set, wherein the image sample data set includes a clean sample data set and a noise sample data set; The clean sample data set includes a clean sample set obtained by screening out a difficult-to-classify sample set from the original data sample, and a difficult-to-identify sample set obtained by screening the difficult-to-classify sample set through a noise recognition model; The noise sample data set is composed of a noise sample set obtained after the difficult-to-classify sample set is screened by the noise recognition model, and the difficult-to-classify sample set is obtained by iteratively screening the original data sample set through a multi-model interaction algorithm.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 4 is implemented.
7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.
8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.
Citation Information
Cited By
Smooth noise injection-based intelligent model robustness test method and system, medium and equipment
CN121211004A
