Image processing method and device, electronic equipment and storage medium

Abstract

The present application provides an image processing method and device, electronic equipment and storage medium, by taking the first comparison result and the second comparison result as a general judgment basis for determining whether the label of the corresponding target image data is a noise label or a non-noise label, since the first comparison result and the second comparison result only depend on the clustering label and the prediction label, and are independent of the specific application of which data set, therefore, based on the first comparison result and the second comparison result as a general judgment basis for determining whether the label of the corresponding target image data is a noise label or a non-noise label, it can be applied to different application scenarios, and then the prediction model trained by the first type image data set is used to correct the label of the second type image data set after the label removal processing, so as to provide a more accurate corrected label compared with the original noise label as much as possible, and the adverse effects of the noise label can be reduced.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] With the emergence of large-scale datasets in recent years, deep neural networks (DNNs) have been applied to many machine learning tasks, such as computer vision, information retrieval, and language processing. Their success depends on the availability of large amounts of labeled data. Unreliable labels are called noisy labels. In the presence of noisy labels, the large number of model parameters causes the DNN to overfit. Training of DNNs is susceptible to the influence of noisy labels. DNNs can easily fit the entire training dataset with any proportion of noisy labels, ultimately resulting in poor generalization ability on the test dataset. Therefore, mitigating the adverse effects of noisy labels is crucial.

[0003] However, the image processing methods in the prior art often rely on the type of image datasets to identify noise labels in image datasets, and therefore have poor generalization and cannot be applied to different application scenarios. Summary of the Invention

[0004] In view of this, the purpose of this application is to propose an image processing method, device, electronic device and storage medium to solve or partially solve the above technical problems.

[0005] Based on the above objectives, the first aspect of the present application provides an image processing method, comprising:

[0006] Obtain the cluster labels corresponding to the target image data in the initial image dataset at different prediction times and the prediction labels corresponding to different prediction times;

[0007] Comparing the cluster labels corresponding to a preset number of adjacent prediction times to obtain a first comparison result, and comparing the prediction labels corresponding to a preset number of adjacent prediction times to obtain a second comparison result;

[0008] In response to the first comparison result being the same and the second comparison result being the same, determining that the label of the target image data is a non-noise label, and combining all target image data labeled with the non-noise label in the initial image dataset as a first type of image dataset;

[0009] In response to the first comparison result being different and / or the second comparison result being different, determining that the label of the target image data is a noise label, and combining all target image data labeled with the noise label in the initial image dataset as a second type of image dataset;

[0010] Using the first type of image dataset to train a pre-built initial model to obtain a prediction model;

[0011] The second type of image data set is subjected to label removal processing, and the label correction is performed on the second type of image data set after the label removal processing using the prediction model to obtain a correction label corresponding to the second type of image data set.

[0012] Optionally, performing label correction on the second type of image dataset that has undergone label removal processing using the prediction model to obtain a corrected label corresponding to the second type of image dataset includes:

[0013] Performing strong enhancement processing on the second type image dataset that has undergone label removal processing to obtain a second type image dataset that has undergone strong enhancement processing, and performing weak enhancement processing on the second type image dataset that has undergone label removal processing to obtain a second type image dataset that has undergone weak enhancement processing;

[0014] Inputting the second type of image data set that has undergone strong enhancement processing into the prediction model, outputting a first prediction result through the prediction model, and inputting the second type of image data set that has undergone weak enhancement processing into the prediction model, outputting a second prediction result through the prediction model;

[0015] constructing a first loss function based on the first prediction result and the second prediction result, performing a minimization process on the first loss function, and training and adjusting the prediction model according to a result of minimizing the first loss function during the minimization process to obtain a trained prediction model;

[0016] The second type of image data set that has undergone label removal processing is input into the trained prediction model to obtain a correction label corresponding to the second type of image data set.

[0017] Optionally, obtaining cluster labels corresponding to target image data in the initial image dataset at different prediction times includes:

[0018] Obtaining initial labels of target image data in the initial image dataset;

[0019] Performing a difference process using the initial label of the target image data and the predicted labels corresponding to different prediction times to obtain loss values ​​corresponding to different prediction times;

[0020] The k-means clustering algorithm is used to cluster the loss values ​​corresponding to different prediction times, and the cluster labels corresponding to the target image data in the initial image dataset at different prediction times are obtained.

[0021] Optionally, the using the first type of image dataset to train a pre-built initial model to obtain a prediction model includes:

[0022] Obtain initial labels for the first type of image dataset and build an initial model;

[0023] Inputting the first type of image data set into the initial model, and outputting a training prediction label through the initial model;

[0024] A second loss function is constructed based on the initial labels of the first type of image data set and the predicted labels for training, and the second loss function is minimized. The initial model is trained and adjusted according to the result of minimizing the second loss function during the minimization process to obtain the prediction model.

[0025] Optionally, the using the first type of image dataset to train a pre-built initial model to obtain a prediction model includes:

[0026] Performing strong enhancement processing on the first type of image dataset to obtain a first type of image dataset that has undergone strong enhancement processing;

[0027] Obtaining initial labels of the first type of image dataset that has undergone strong enhancement processing and building an initial model;

[0028] Inputting the first type of image data set that has undergone strong enhancement processing into the initial model, and outputting a training prediction label through the initial model;

[0029] A third loss function is constructed based on the initial labels of the first type of image data set that has undergone strong enhancement processing and the predicted labels for training, and the third loss function is minimized. The initial model is trained and adjusted according to the result of minimizing the third loss function during the minimization process to obtain the prediction model.

[0030] Optionally, after performing label correction on the second type of image dataset that has undergone label removal processing using the prediction model to obtain a corrected label corresponding to the second type of image dataset, the method further includes:

[0031] counting the number of the second-type image datasets and the number of the initial image datasets, and performing ratio processing on the number of the second-type image datasets and the number of the initial image datasets to obtain a first ratio processing result;

[0032] Determining the number of noise labels in the second type of image data set provided with the correction labels, and performing ratio processing on the number of noise labels and the number of labels in the initial image data set to obtain a second ratio processing result;

[0033] In response to the second ratio processing result being less than the first ratio processing result, determining that the correction label corresponding to the second type of image data set is correct; or,

[0034] In response to the second ratio processing result being greater than or equal to the first ratio processing result, it is determined that the correction label corresponding to the second type of image data set is incorrect.

[0035] Optionally, after performing label correction on the second type of image dataset that has undergone label removal processing using the prediction model to obtain a corrected label corresponding to the second type of image dataset, the method further includes:

[0036] Performing an accuracy test on the prediction model using a preset accuracy test indicator to obtain an accuracy test result;

[0037] In response to the accuracy test result being within a threshold range corresponding to the accuracy test indicator, determining that the correction label corresponding to the second type of image data set is correct; or,

[0038] In response to the accuracy test result not being within a threshold range corresponding to the accuracy test indicator, determining that the correction label corresponding to the second type of image dataset is incorrect, wherein the accuracy test indicator includes at least one of the following:

[0039] Precision index, recall index, F1 score index and intersection-over-union index.

[0040] Based on the same inventive concept, the second aspect of the present application provides an image processing device, comprising:

[0041] An acquisition module is configured to acquire cluster labels corresponding to target image data in the initial image data set at different prediction times and prediction labels corresponding to different prediction times;

[0042] a comparison module configured to compare cluster labels corresponding to a preset number of adjacent prediction times to obtain a first comparison result, and to compare prediction labels corresponding to a preset number of adjacent prediction times to obtain a second comparison result;

[0043] a first combining module configured to, in response to the first comparison result being the same and the second comparison result being the same, determine that the label of the target image data is a non-noise label, and combine all target image data labeled with the non-noise label in the initial image dataset as a first type of image dataset;

[0044] a second combining module configured to, in response to the first comparison result being different and / or the second comparison result being different, determine that the label of the target image data is a noise label, and combine all target image data labeled with the noise label in the initial image dataset as a second type of image dataset;

[0045] a training module configured to train a pre-built initial model using the first type of image dataset to obtain a prediction model;

[0046] The label correction module is configured to perform label removal processing on the second type of image data set, and use the prediction model to perform label correction on the second type of image data set that has undergone label removal processing to obtain a correction label corresponding to the second type of image data set.

[0047] Based on the same inventive concept, the third aspect of this application provides an electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable by the processor, wherein the processor implements the method described in the first aspect above when executing the computer program.

[0048] Based on the same inventive concept, the fourth aspect of the present application provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the method described in the first aspect above.

[0049] From the above description, it can be seen that the image processing method, device, electronic device and storage medium provided by the present application obtain a first comparison result by comparing the cluster labels corresponding to a preset number of adjacent prediction times, and obtain a second comparison result by comparing the cluster labels corresponding to a preset number of adjacent prediction times, and then combine the first comparison result and the second comparison result as a general judgment basis for determining whether the label of the corresponding target image data is a noise label or a non-noise label. Since the first comparison result and the second comparison result only depend on the cluster label and the prediction label, and have nothing to do with which specific data set is applicable, the first comparison result and the second comparison result are used as a general judgment basis for whether the label of the corresponding target image data is a noise label or a non-noise label, which can be applied to different application scenarios. Afterwards, the prediction model trained by the first type of image data set is used to perform label correction on the second type of image data set that has been processed by removing the labels, so as to provide a correction label that is more accurate than the original noise label as much as possible, which can reduce the adverse effects of the noise label. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in this application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0051] Figure 1 This is a flowchart of an image processing method according to an embodiment of the present application;

[0052] Figure 2A This is a schematic diagram of the semi-supervised learning process of an embodiment of the present application;

[0053] Figure 2B This is a schematic diagram of the image processing flow in an embodiment of the present application;

[0054] Figure 3 This is a structural block diagram of an image processing device according to an embodiment of the present application;

[0055] Figure 4 A schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the objectives, technical solutions and advantages of this application more clear, this application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0057] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should have the usual meanings understood by people with ordinary skills in the field to which this application belongs. The "first", "second" and similar words used in the embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0058] It is understandable that before using the technical solutions of each embodiment of this application, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0059] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. Thus, the user can independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operation of the technical solution of this application based on the prompt message.

[0060] As an optional but non-limiting implementation, in response to a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0061] It is understandable that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this application. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this application.

[0062] With the emergence of large-scale datasets in recent years, deep neural networks (DNNs) have been applied to many machine learning tasks, such as computer vision, information retrieval, and language processing. Their success depends on the availability of large amounts of labeled data. Unreliable labels are called noisy labels. In the presence of noisy labels, the large number of model parameters causes the DNN to overfit. Training of DNNs is susceptible to the influence of noisy labels. DNNs can easily fit the entire training dataset with any proportion of noisy labels, ultimately resulting in poor generalization ability on the test dataset. Therefore, mitigating the adverse effects of noisy labels is crucial.

[0063] However, existing regularization techniques, such as data augmentation, weight decay, random dropout, and batch normalization, do not completely overcome the overfitting problem on their own. In addition, the accuracy drop from label noise is considered to be more harmful than other noise (such as input noise). Therefore, achieving good generalization ability in the presence of noisy labels is a key issue.

[0064] If traditional machine learning techniques are used to solve the label noise problem, these methods can be roughly divided into four categories: data cleaning, loss proxy, statistical probability, and machine learning model-based.

[0065] (1) Data cleaning

[0066] Training data is cleaned by excluding examples whose labels may be corrupted. Bootstrap aggregating (bagging) and boosting algorithms are used to filter out incorrectly labeled examples to remove highly weighted examples, as incorrectly labeled examples tend to have higher weights than true labeled examples. In addition, various methods such as k-nearest neighbors, outlier detection, and anomaly detection have been widely used to exclude incorrectly labeled examples from noisy training data. However, these methods suffer from excessive cleaning, even leading to excessive removal of truly labeled examples.

[0067] (2) Agency losses

[0068] Inspired by the noise tolerance of the 0-1 loss function, attempts have been made to address its inherent limitations, such as computational hardness and non-convexity, which preclude the application of gradient methods. Consequently, several convex surrogate loss functions that approximate the 0-1 loss function have been proposed to train classifiers specified in the binary classification setting. However, these loss functions do not support multi-class classification tasks.

[0069] (3) Statistical probability

[0070] Assuming that feature distributions help solve the problem of learning from noisy labels, clustering is used to estimate the confidence of each label, which is then used to weight the training scheme. This confidence is also used to convert hard labels into soft labels to reflect the uncertainty of the labels. In addition to these clustering methods, Bayesian methods for graphical models have been proposed so that they can benefit from using any type of prior information during the learning process. However, this series of methods may exacerbate the overfitting problem due to the increase in the number of model parameters.

[0071] (4) Methods based on machine learning models

[0072] These methods include penalizing misclassified examples during learning and extending decision tree models with new segmentation criteria to address the overfitting problem when training data is not completely reliable. However, applying their design principles to deep learning is not feasible.

[0073] The noisy training methods based on deep learning mainly include the following four methods: specific structure design, robustness self-regulation, loss adjustment, and sample selection.

[0074] (1) Specific structural design

[0075] This class of methods aims to improve model robustness by designing a new network architecture to mimic the noise transition matrix of a noisy dataset. For example, a noise adaptation layer is added on top of a normalized exponential function (SoftMax) layer and a new specialized architecture is designed to produce better generalization by modifying the output of a deep neural network based on the estimated label transition probabilities. Furthermore, sophisticated network structures can be designed for specific models with the goal of increasing the reliability of the estimated noise transition matrix. These methods can handle a variety of noise types but are not easily extended to other general network structures.

[0076] (2) Robust regularization

[0077] Regularization is a commonly used method to mitigate overfitting, but it can only handle moderate levels of noise. When the noise is very severe, the network still achieves poor generalization performance. Therefore, many more advanced regularization techniques have been proposed, including explicit and implicit regularization. Explicit regularization involves directly modifying the loss, such as weight decay and random dropout. This usually requires the introduction of model hyperparameters, but the expanded feature and label space slows down convergence.

[0078] (3) Loss adjustment

[0079] These methods aim to make traditional optimization processes more robust to noise. They minimize the negative impact of noisy labels by adjusting the update rules during training. However, they can overcorrect, especially when there are a large number of classes or mislabeled examples. Key methods include loss correction, loss reweighting, and label inversion. Loss correction modifies the loss by multiplying the model output by an estimated label transfer matrix. These methods rely heavily on estimating the noisy transfer matrix and therefore typically require prior knowledge (such as anchors or a clean validation set). Loss reweighting assigns less weight to false positives and more weight to true positives when calculating the loss. These methods rely on manually designed weight functions and hyperparameters, making them difficult to implement in practice. Label inversion combines the network's output labels with noisy labels as the refurbished labels. Unlike loss correction and reweighting, these methods explicitly replace all noisy labels with approximate clean labels. However, when the proportion of noisy labels is high, they can easily overfit to the incorrectly refurbished examples.

[0080] (4) Sample selection

[0081] Sample selection uses only the clean samples selected in each batch to train the network. The memory properties of deep neural networks are exploited to re-extract clean samples from noisy datasets. Deep neural networks tend to first learn simple and general patterns and then gradually overfit to the noise. Therefore, they often select samples with low training loss as clean samples. A drawback is that incorrect sample selection can lead to cumulative errors. Therefore, many methods utilize multiple networks to collaborate or train for multiple rounds. Key approaches include multi-network training, multi-round training, and hybrid models. Multi-network training typically involves two networks training together, updating only when there are disagreements in sample selection. Mutual supervision can also be used to reduce differences between the two networks and achieve similar predictions. Multi-round training does not require additional networks. This method continuously optimizes the selected clean training set through repeated iterations. A drawback is that the computational cost increases linearly with the number of training rounds. A significant drawback of sample selection is that all unselected samples are discarded. To fully utilize noisy data, hybrid strategies have emerged. These methods often incorporate semi-supervised learning. First, the noisy data is partitioned into labeled and unlabeled groups, and then semi-supervised learning methods are applied to train the partitioned data. The disadvantage is that there are too many hyperparameters and it inevitably introduces additional computational overhead.

[0082] Compared to structure-specific designs, methods based on loss adjustment and sample selection are more versatile and can improve the generalization performance of the model. Therefore, currently effective denoising training methods generally use these two techniques to address dataset noise.

[0083] Loss adjustment can be generally divided into two categories. One type of correction method explicitly or implicitly processes all samples and corrects the loss by relabeling noisy samples. For relabeling methods, noisy samples are modeled using directed graphical models, conditional random fields, knowledge graphs, or deep neural networks. However, they require access to a small set of clean samples first. The other type of correction method focuses on reweighting training samples or separating clean and noisy samples, thereby combining a teacher academic model to guide the student network by assigning weights to samples and maintaining consistency between the two networks by updating the divergent data.

[0084] Sample selection is often combined with semi-supervised learning techniques, which aim to improve model performance by leveraging unlabeled data. Current state-of-the-art semi-supervised learning (SSL) methods primarily involve adding an additional loss term to regularize training on unlabeled data. Regularization can be categorized into two types: consistency regularization and entropy minimization. Consistency regularization forces the model to produce consistent predictions across augmented input data, while entropy minimization encourages the model to make high-confidence predictions on unlabeled data.

[0085] These methods make full use of the characteristics of noisy data and the advantages of deep learning, integrating feature learning with deep learning methods. Based on reducing the noise rate of the original dataset, they provide high-quality input for the training of deep network models, thereby improving the generalization and accuracy of the models.

[0086] However, existing noisy training methods also have some problems that cannot be ignored. First, in the sample selection process, different technical means are used to divide the data set, and there is still a large amount of noise data that has not been screened out. These data will still be regarded as clean samples and sent to the network for training, which will undoubtedly affect the training effect of the network model. Secondly, the current discovery and identification of noise data characteristics are still unclear, and there is no universal indicator as a basic basis for judging noise samples. This makes it impossible to migrate from one scenario to other scenarios. Finally, model training and dataset denoising processing are different angles to improve the training effect of deep learning. How to better combine the two is also a problem that needs to be solved urgently.

[0087] An embodiment of the present application provides an image processing method, which obtains a first comparison result by comparing cluster labels corresponding to a preset number of adjacent prediction times, and obtains a second comparison result by comparing cluster labels corresponding to a preset number of adjacent prediction times, and then combines the first comparison result and the second comparison result as a general judgment basis for determining whether the label of the corresponding target image data is a noise label or a non-noise label. Since the first comparison result and the second comparison result only depend on the cluster label and the prediction label, and are not related to which specific data set is applicable, the first comparison result and the second comparison result are used as a general judgment basis for whether the label of the corresponding target image data is a noise label or a non-noise label, which can be applied to different application scenarios. Thereafter, the prediction model trained by the first type of image data set is used to perform label correction on the second type of image data set that has been processed by removing the labels, so as to provide a correction label that is more accurate than the original noise label as much as possible, which can reduce the adverse effects of the noise label.

[0088] like Figure 1 As shown, the method of this embodiment includes:

[0089] Step 101: Obtain cluster labels corresponding to target image data in an initial image dataset at different prediction times and prediction labels corresponding to different prediction times.

[0090] In this step, the cluster label is used to indicate whether the label of the target image data belongs to the noise label.

[0091] Use image acquisition equipment (for example, PC camera, industrial camera, surveillance camera, scanner, or video camera) to capture images in corresponding application scenarios to obtain an initial image dataset.

[0092] Application scenarios include road crack detection scenarios, medical imaging application scenarios, or image semantic classification scenarios.

[0093] For example, the application scenario is a pavement crack detection scenario. A rapid road condition acquisition device (CiCS I) is used to collect a pavement image dataset. The pavement image dataset includes multiple pavement image data. Any pavement image data among the multiple pavement image data is used as the target image data. For the target image data, the cluster labels corresponding to different prediction times and the prediction labels corresponding to different prediction times are obtained.

[0094] Since the division of cluster labels corresponding to a single round of predictions often performs poorly, it is necessary to obtain the cluster labels corresponding to the target image data in the initial image dataset at different prediction times, so as to integrate multiple rounds of cluster labels to improve robustness and accuracy.

[0095] In order to give full play to the results of multiple rounds of prediction, it is also necessary to obtain the prediction labels corresponding to the target image data in the initial image dataset at different prediction times on the basis of clustering labels based on different prediction times, so as to evaluate the possibility that the labels of the target image data in the initial image dataset are noise labels.

[0096] Step 102 : Compare the cluster labels corresponding to a preset number of adjacent prediction times to obtain a first comparison result, and compare the prediction labels corresponding to a preset number of adjacent prediction times to obtain a second comparison result.

[0097] In this step, the cluster labels corresponding to a preset number of adjacent prediction times are integrated, combined with the prediction labels corresponding to the preset number of adjacent prediction times, to ensure that almost all target image data labeled as noise labels in the initial image data set are identified.

[0098] The first comparison result is used to indicate whether the cluster labels corresponding to the integrated multiple rounds of predictions are the same, and the second comparison result is used to indicate whether the prediction labels corresponding to the integrated multiple rounds of predictions are the same.

[0099] This application uses a general judgment basis for whether the label corresponding to the target image data is a noise label or a non-noise label. Since the cluster labels and prediction labels involved in the first comparison result and the second comparison result are obtained from the division process, they are independent of the type of data set and have nothing to do with which specific data set is applicable. This also shows the versatility of using the first comparison result and the second comparison result for judgment, and can be applied to different application scenarios.

[0100] Step 103: In response to the first comparison result being the same and the second comparison result being the same, determining that the label of the target image data is a non-noise label, and combining all target image data labeled with a non-noise label in the initial image data set as a first type of image data set.

[0101] In this step, the first type of image data set represents a set of image data whose labels are non-noise labels.

[0102] When the first comparison result and the second comparison result are the same, it indicates that the cluster labels corresponding to multiple rounds of prediction are the same, and the prediction labels corresponding to multiple rounds of prediction are also the same. At this time, the label of the corresponding target image data is a non-noise label. In this way, all target image data labeled with non-noise labels in the initial image data set are combined to form a first type of image data set.

[0103] Through the first comparison result and the second comparison result, it can be quickly determined whether the label of the corresponding target image data is a noise label or a non-noise label.

[0104] Step 104: In response to the first comparison result being different and / or the second comparison result being different, determining that the label of the target image data is a noise label, and combining all target image data labeled with noise labels in the initial image dataset as a second type of image dataset.

[0105] In this step, the second type of image data set represents a set of image data whose labels are noise labels.

[0106] When the first comparison result and the second comparison result are both different, or the first comparison result is the same and the second comparison result is different, or the first comparison result is different and the second comparison result is the same, it indicates that the clustering labels corresponding to multiple rounds of prediction are different, and / or the prediction labels corresponding to multiple rounds of prediction are different. At this time, the label of the corresponding target image data is a noise label. In this way, all target image data labeled with noise labels in the initial image data set are combined to form a second type of image data set.

[0107] Through the first comparison result and the second comparison result, it can be quickly determined whether the label of the corresponding target image data is a noise label or a non-noise label.

[0108] Step 105: Use the first type of image dataset to train the pre-built initial model to obtain a prediction model.

[0109] In this step, since the labels corresponding to the first type of image dataset are all non-noise labels, using the first type of image dataset to train the pre-built initial model can ensure the accuracy of the initial model training.

[0110] Step 106: perform label removal processing on the second type of image dataset, and use the prediction model to perform label correction on the second type of image dataset that has undergone label removal processing to obtain a corrected label corresponding to the second type of image dataset.

[0111] In this step, the original labels of the second type of image dataset are discarded, and the prediction model is used to perform label correction on the second type of image dataset that has discarded the original labels to make a reliable guess about its label, thereby providing a corrected label that is more accurate than the original noise label as much as possible. Using this label as its true label can reduce the adverse effects of the noise label.

[0112] Through the above scheme, the cluster labels corresponding to the preset number of adjacent prediction times are compared to obtain a first comparison result, and the cluster labels corresponding to the preset number of adjacent prediction times are compared to obtain a second comparison result. Then, the first comparison result and the second comparison result are combined as a general judgment basis for determining whether the label of the corresponding target image data is a noise label or a non-noise label. Since the first comparison result and the second comparison result only depend on the cluster label and the prediction label, and have nothing to do with which specific data set is applicable, the first comparison result and the second comparison result are used as a general judgment basis for whether the label of the corresponding target image data is a noise label or a non-noise label. It can be applied to different application scenarios. Afterwards, the prediction model trained by the first type of image data set is used to perform label correction on the second type of image data set that has been processed by removing the labels, so as to provide a correction label that is more accurate than the original noise label as much as possible, which can reduce the adverse effects of the noise label.

[0113] In some embodiments, in step 106, using the prediction model to perform label correction on the second type of image dataset that has undergone label removal processing to obtain a corrected label corresponding to the second type of image dataset includes:

[0114] In step A1, strong enhancement processing is performed on the second type image dataset that has undergone label removal processing to obtain a second type image dataset that has undergone strong enhancement processing, and weak enhancement processing is performed on the second type image dataset that has undergone label removal processing to obtain a second type image dataset that has undergone weak enhancement processing.

[0115] In step A2, the second type of image data set that has undergone strong enhancement processing is input into the prediction model, and the prediction model outputs a first prediction result, and the second type of image data set that has undergone weak enhancement processing is input into the prediction model, and the prediction model outputs a second prediction result.

[0116] Step A3: construct a first loss function based on the first prediction result and the second prediction result, and minimize the first loss function. According to the result of minimizing the first loss function during the minimization process, the prediction model is trained and adjusted to obtain a trained prediction model.

[0117] Step A4: input the second type of image data set that has undergone label removal processing into the trained prediction model to obtain a correction label corresponding to the second type of image data set.

[0118] In the above scheme, two semi-supervised learning methods are used for label correction: consistency regularization and pseudo labeling.

[0119] like Figure 2AAs shown, the prediction model is obtained by training the initial model through a first type of labeled image data set, and then the second type of unlabeled image data set is weakly enhanced to obtain a second type of image data set that has been weakly enhanced, and is also strong enhanced to obtain a second type of image data set that has been weakly enhanced, so as to achieve the purpose of expanding the amount of data.

[0120] The second type of image data set that has undergone strong enhancement processing is input into the prediction model, and the prediction model outputs a first prediction result. The second type of image data set that has undergone weak enhancement processing is input into the prediction model, and the prediction model outputs a second prediction result. Through the consistency regularization method, the first loss function constructed based on the first prediction result and the second prediction result is minimized. According to the result of minimizing the first loss function in the minimization process, the prediction model is trained and adjusted to obtain a trained prediction model, so that the trained prediction model is more accurate. At this time, the pseudo-label method is used to use the trained prediction model to predict the second type of image data set without labels, which can provide a correction label that is more accurate than the original noise label as much as possible, thereby reducing the adverse effects of the noise label.

[0121] In some embodiments, in step 101, obtaining cluster labels corresponding to target image data in the initial image dataset at different prediction times includes:

[0122] Step B1: Obtain the initial label of the target image data in the initial image dataset.

[0123] Step B2: performing difference processing on the initial label of the target image data and the prediction label corresponding to different prediction times to obtain loss values ​​corresponding to different prediction times.

[0124] Step B3: Use the k-means clustering algorithm to cluster the loss values ​​corresponding to different prediction times to obtain the cluster labels corresponding to the target image data in the initial image data set under different prediction times.

[0125] In the above scheme, the k-means clustering algorithm (K-means) is used to cluster the target image data in the initial image data set and the loss values ​​corresponding to different prediction times, so that the initial image data set is divided into two categories. The labels carried by the target image data with smaller losses are identified as non-noise labels, and the labels carried by the target image data with larger losses are identified as noise labels.

[0126] For example, when the cluster label is 0, it indicates a low loss value. At this time, the cluster label indicates that the label of the corresponding target image data is identified as a non-noise label. When the cluster label is 1, it indicates a high loss value. At this time, the cluster label indicates that the label of the corresponding target image data is identified as a noise label.

[0127] The k-means clustering algorithm can be used to simply and efficiently obtain the cluster labels corresponding to the target image data in the initial image dataset under different prediction times.

[0128] In some embodiments, step 105 includes:

[0129] Step C1: Obtain initial labels for the first type of image dataset and build an initial model.

[0130] Step C2: input the first type of image data set into the initial model, and output a training prediction label through the initial model.

[0131] Step C3: construct a second loss function based on the initial labels of the first type of image data set and the predicted labels for training, and minimize the second loss function. According to the result of minimizing the second loss function during the minimization process, the initial model is trained and adjusted to obtain the prediction model.

[0132] In the above scheme, after the initial image dataset is divided into a first type of image dataset and a second type of image dataset, the initial model is trained using the first type of image dataset with labels to obtain a prediction model, so that the prediction model can be further trained using the second type of image dataset without labels, so that the trained prediction model can be more accurate.

[0133] In some embodiments, step 105 includes:

[0134] Step D1: performing strong enhancement processing on the first type image dataset to obtain a first type image dataset that has undergone strong enhancement processing.

[0135] Step D2: Obtain the initial labels of the first type of image dataset that has undergone strong enhancement processing and build an initial model.

[0136] In step D3, the first type of image data set that has undergone strong enhancement processing is input into the initial model, and the initial model outputs a prediction label for training.

[0137] Step D4, constructing a third loss function based on the initial labels of the first type of image data set that has undergone strong enhancement processing and the predicted labels for training, and minimizing the third loss function, and training and adjusting the initial model according to the result of minimizing the third loss function during the minimization process to obtain the prediction model.

[0138] In the above scheme, in order to ensure the accuracy of the initial model training, it is necessary to expand the number of first type image data sets. This application adopts a method of performing strong enhancement processing on the first type image data set to obtain a first type image data set that has undergone strong enhancement processing.

[0139] By using the strong enhancement processing method, the diversity of the first type of image dataset can be increased without changing the category of the first type of image dataset, thereby improving the robustness of the prediction model.

[0140] In some embodiments, after step 106, the method further includes:

[0141] Step E1: Count the number of the second type image data sets and the number of the initial image data sets, and perform ratio processing on the number of the second type image data sets and the number of the initial image data sets to obtain a first ratio processing result.

[0142] Step E2: determining the number of noise labels in the second type of image data set with the correction labels, and performing ratio processing on the number of noise labels and the number of the initial image data set to obtain a second ratio processing result.

[0143] Step E3: In response to the second ratio processing result being less than the first ratio processing result, determining that the correction label corresponding to the second type of image data set is correct. Or,

[0144] Step E4: In response to the second ratio processing result being greater than or equal to the first ratio processing result, determining that the correction label corresponding to the second type of image data set is incorrect.

[0145] In the above scheme, in order to test the accuracy of dividing the initial image dataset into the first type of image dataset and the second image dataset, that is, after the original initial image dataset is divided into the first type of image dataset with non-noise labels and the second type of image dataset with discarded noise labels, after label correction processing, whether the proportion of noise labels in the second type of image dataset with corrected labels that belong to noise labels is reduced compared with the original initial image dataset.

[0146] When the second ratio processing result is less than the first ratio processing result, it is determined that the correction label corresponding to the second type of image data set is correct, or when the second ratio processing result is less than the first ratio processing result, it is determined that the correction label corresponding to the second type of image data set is correct.

[0147] The accuracy of dividing the initial image dataset into the first type of image dataset and the second image dataset can be quickly determined by whether the correction labels corresponding to the second type of image dataset are accurate, by determining whether the proportion of noise labels in the second type of image dataset that are noise labels is reduced compared to the original initial image dataset.

[0148] In some embodiments, after step 106, the method further includes:

[0149] Step F1, performing an accuracy test on the prediction model using a preset accuracy test index to obtain an accuracy test result.

[0150] Step F2: In response to the accuracy test result being within a threshold range corresponding to the accuracy test indicator, determining that the correction label corresponding to the second type of image data set is correct. Or,

[0151] Step F3: In response to the accuracy test result not being within a threshold range corresponding to the accuracy test indicator, determining that the correction label corresponding to the second type of image dataset is incorrect, wherein the accuracy test indicator includes at least one of the following:

[0152] Precision index, recall index, F1 score index and intersection-over-union index.

[0153] In the above scheme, in order to test the accuracy of the division of the initial image dataset into the first type of image dataset and the second image dataset, that is, after the original initial image dataset is divided into the first type of image dataset with non-noise labels and the second type of image dataset with the noise labels discarded, the preset accuracy test index corresponding to the prediction model is used to determine whether it is improved compared with other experimental methods, so as to quickly determine the accuracy of the division of the initial image dataset into the first type of image dataset and the second image dataset by whether the correction label corresponding to the second type of image dataset is accurate.

[0154] Alternatively, the accuracy of the initial image dataset being divided into the first type of image dataset and the second image dataset can be quickly determined by whether the preset accuracy test indicator corresponding to the prediction model is within the threshold range corresponding to the accuracy test indicator.

[0155] The accuracy test indicators include at least one of the following:

[0156] Precision index, recall index, F1 score index (F1 score) and intersection over union index (IoU).

[0157] Based on the same inventive concept, the application scenarios corresponding to the image processing methods of the above embodiments are described in detail. Figure 2B As shown, the details are as follows:

[0158] Sample division stage: Assume that the number of predictions (epochs) is m and the number of ensembles (ensembles) is 3.

[0159] 1. Each sample gets a loss value and a predicted label after one prediction epoch.

[0160] 2. In the i-th prediction, integrate the three prediction epochs i-1 and i-2. If a sample has low loss in all three prediction epochs and the predicted labels are basically the same, then in this round of integration, its label is considered a non-noise label, otherwise it is a noise label.

[0161] 3. Since three prediction epochs are integrated once, m prediction epochs will be integrated m-2 times. Before training, the partitioning results of the last round of prediction epochs are fed into model training.

[0162] (1) Obtaining the original dataset

[0163] The engineering dataset involved in this application (i.e., the initial image dataset D) was screened from a massive amount of actual road surface datasets. The acquisition equipment used was the rapid road condition acquisition equipment (CiCS I). The image size obtained by screening was about 2000×3000 pixels, with a total of 50,000 images, of which the ratio of image data containing cracks to image data not containing cracks was 1:1. Whether it is the type of cracks (horizontal cracks, longitudinal cracks, reticular cracks, white cracks, and fuzzy cracks, etc.) or the presence of various interference factors (lighting, paint stains, repair marks, and signs and markings, etc.), the situation in the actual engineering road surface images is particularly complicated. Under such a complex image background, the amount of work required to annotate pixel-level labels is huge. In order to effectively save manpower and material costs, 100×100 pixel grids are chosen instead of single pixel values ​​as the basic recognition unit for manual annotation. To facilitate calculations, the image size is standardized to 2200 × 3400 pixels. A single image corresponds to a label matrix of size 22 × 34. The value of each matrix point is 0 or 1. 1 indicates that there is a crack in the corresponding 100 × 100 pixel area in the original image represented by the matrix point, and 0 indicates that there is no crack in the corresponding 100 × 100 pixel area in the original image represented by the matrix point.

[0164] (2) Collaborative training

[0165] Using two models (NetA) for collaborative training effectively avoids confirmation bias caused by training a single model. The collaborative training process primarily involves dividing the training data between one network and feeding it to the other network after each prediction epoch. One network is trained at a time while the other remains fixed. Due to different (random) parameter initializations, different training data partitions, different (random) mini-batch sequences, and different training objectives, the two networks diverge from each other. This divergence provides the two networks with different abilities to filter out different types of errors, making the model more robust to noise.

[0166] (3) Multi-round training integration 1: loss modeling and clustering integration

[0167] Compared with target image data with noisy labels, image data with non-noisy labels are more suitable for training deep networks, resulting in lower loss values. In each round of training, the loss values ​​of 50,000 samples can be obtained. Then, the k-means clustering algorithm (k-means) is used to perform binary clustering on the 50,000 loss values, dividing them into a class with high loss values ​​and a class with low loss values. The cluster label of each target image data is 0 or 1, 0 represents a low loss value, and 1 represents a high loss value. After one round of training, the 50,000 target image data have their own predicted labels, assuming L i, where i is the index of the corresponding target image data. When the integration starts, a comprehensive judgment is made based on the cluster labels corresponding to the target image data of the current round and the previous rounds. If there is only one target image data whose cluster label is 0 in each round, it means that it has maintained a low loss state during the training process, then it can be determined that the label of the target image data is a non-noise label. Otherwise, it is considered a noise label. Assuming that the number of integrations (that is, the preset number of adjacent predictions) is 3, then the division result of the i-th target image data in the a-th round depends on the three cluster labels of the a-2, a-1, and a-th rounds. Only when all three cluster labels are 0, is it determined that the label of the target image data is preliminarily divided into a non-noise label.

[0168] (4) Multi-round training integration 2: prediction label integration

[0169] Similar to loss clustering integration, the predicted label values ​​of samples are collected during each round of training, assuming that P i , where i is the index of the corresponding target image data. When the integration starts, a comprehensive judgment is made by combining the predicted labels of the current round and the previous rounds. Only when the predicted labels of the target image data of these rounds are the same, then it can be determined that the label with the target image data is a non-noise label. Otherwise, it is considered a noise label. Assuming the number of integrations is 3, the division result of the i-th sample in the a-th round depends on the three predicted labels of the a-2, a-1, and a-th rounds. Only when these three predicted labels are the same, the label of the target image data is determined to be a non-noise label.

[0170] In each partition, not only loss clustering integration is used, but also predicted label integration. Since each target image data will have a cluster label (0 or 1) and a predicted label after a round of training, this means that when the number of integrations is 3, we use the 6 values ​​of each target image data in the current round, the previous round, and the previous round: Only when and Only when , the label of the corresponding target image data is classified as a non-noise label, otherwise it is classified as a noise label. This is also an evaluation indicator established to judge non-noise labels and noise labels.

[0171] Target image data that is considered to have a non-noise label retains its original label, while target image data that is considered to have a noise label directly discards its original label. Thus, the result of a single partition is obtained, preliminarily dividing the entire initial image dataset into two parts: a first type of image dataset (x) containing non-noise labels and a second type of image dataset (u) discarding noise labels.

[0172] (5) Pseudo-labeling technology corrects incorrect labels

[0173] During the semi-supervised learning phase, pre-matching (PreMatch) is improved through label co-refinement and joint guessing to account for label noise. For the first type of image dataset with non-noisy labels, strong augmentation is used to augment the data, and all predictions from both networks are used to jointly refine their true labels. For the second type of image dataset with noisy labels, weak augmentation is used to augment the data, directly discarding the original labels. The two networks are then combined to make a reliable guess for their labels, which is used as the true label.

[0174] (6) Result evaluation

[0175] The evaluation of the entire technical solution includes the accuracy of the original dataset partitioning. Specifically, after the original dataset is divided into a first-type image dataset (x) containing non-noisy labels and a second-type image dataset (u) discarding noisy labels, is the percentage of noisy labels in the second-type image dataset with corrected labels reduced compared to the original initial image dataset after label correction? Another evaluation angle is to see whether the accuracy, recall rate, F1 score, and intersection over union (IoU) of the prediction model trained on the partitioned dataset are improved compared to other experimental methods.

[0176] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied in a distributed scenario and performed by multiple devices working together. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the method.

[0177] It should be noted that the above description is limited to some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0178] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides an image processing device.

[0179] refer to Figure 3, the image processing device comprises:

[0180] An acquisition module 301 is configured to acquire cluster labels corresponding to target image data in an initial image dataset at different prediction times and prediction labels corresponding to different prediction times;

[0181] The comparison module 302 is configured to compare the cluster labels corresponding to a preset number of adjacent prediction times to obtain a first comparison result, and compare the prediction labels corresponding to a preset number of adjacent prediction times to obtain a second comparison result;

[0182] A first combining module 303 is configured to, in response to the first comparison result being the same and the second comparison result being the same, determine that the label of the target image data is a non-noise label, and combine all target image data labeled with the non-noise label in the initial image dataset as a first type of image dataset;

[0183] The second combining module 304 is configured to, in response to the first comparison result being different and / or the second comparison result being different, determine that the label of the target image data is a noise label, and combine all target image data labeled with the noise label in the initial image dataset as a second type of image dataset;

[0184] The training module 305 is configured to train a pre-built initial model using the first type of image dataset to obtain a prediction model;

[0185] The label correction module 306 is configured to perform label removal processing on the second type of image dataset, and use the prediction model to perform label correction on the second type of image dataset that has undergone label removal processing to obtain a correction label corresponding to the second type of image dataset.

[0186] In some embodiments, the label correction module 306 is specifically configured to:

[0187] Performing strong enhancement processing on the second type image dataset that has undergone label removal processing to obtain a second type image dataset that has undergone strong enhancement processing, and performing weak enhancement processing on the second type image dataset that has undergone label removal processing to obtain a second type image dataset that has undergone weak enhancement processing;

[0188] Inputting the second type of image data set that has undergone strong enhancement processing into the prediction model, outputting a first prediction result through the prediction model, and inputting the second type of image data set that has undergone weak enhancement processing into the prediction model, outputting a second prediction result through the prediction model;

[0189] constructing a first loss function based on the first prediction result and the second prediction result, performing a minimization process on the first loss function, and training and adjusting the prediction model according to a result of minimizing the first loss function during the minimization process to obtain a trained prediction model;

[0190] The second type of image data set that has undergone label removal processing is input into the trained prediction model to obtain a correction label corresponding to the second type of image data set.

[0191] In some embodiments, the acquisition module 301 is specifically configured to:

[0192] Obtaining initial labels of target image data in the initial image dataset;

[0193] Performing a difference process using the initial label of the target image data and the predicted labels corresponding to different prediction times to obtain loss values ​​corresponding to different prediction times;

[0194] The k-means clustering algorithm is used to cluster the loss values ​​corresponding to different prediction times, and the cluster labels corresponding to the target image data in the initial image dataset at different prediction times are obtained.

[0195] In some embodiments, the training module 305 is specifically configured to:

[0196] Obtain initial labels for the first type of image dataset and build an initial model;

[0197] Inputting the first type of image data set into the initial model, and outputting a training prediction label through the initial model;

[0198] A second loss function is constructed based on the initial labels of the first type of image data set and the predicted labels for training, and the second loss function is minimized. The initial model is trained and adjusted according to the result of minimizing the second loss function during the minimization process to obtain the prediction model.

[0199] In some embodiments, the training module 305 is specifically configured to:

[0200] Performing strong enhancement processing on the first type of image dataset to obtain a first type of image dataset that has undergone strong enhancement processing;

[0201] Obtaining initial labels of the first type of image dataset that has undergone strong enhancement processing and building an initial model;

[0202] Inputting the first type of image data set that has undergone strong enhancement processing into the initial model, and outputting a training prediction label through the initial model;

[0203] A third loss function is constructed based on the initial labels of the first type of image data set that has undergone strong enhancement processing and the predicted labels for training, and the third loss function is minimized. The initial model is trained and adjusted according to the result of minimizing the third loss function during the minimization process to obtain the prediction model.

[0204] In some embodiments, the image processing apparatus further includes an accuracy determination module, which uses the prediction model to perform label correction on the second type of image dataset that has undergone label removal processing. After obtaining a corrected label corresponding to the second type of image dataset, the accuracy determination module is specifically configured to:

[0205] counting the number of the second-type image datasets and the number of the initial image datasets, and performing ratio processing on the number of the second-type image datasets and the number of the initial image datasets to obtain a first ratio processing result;

[0206] Determining the number of noise labels in the second type of image data set provided with the correction labels, and performing ratio processing on the number of noise labels and the number of labels in the initial image data set to obtain a second ratio processing result;

[0207] In response to the second ratio processing result being less than the first ratio processing result, determining that the correction label corresponding to the second type of image data set is correct; or,

[0208] In response to the second ratio processing result being greater than or equal to the first ratio processing result, it is determined that the correction label corresponding to the second type of image data set is incorrect.

[0209] In some embodiments, the image processing apparatus further includes an accuracy determination module, which uses the prediction model to perform label correction on the second type of image dataset that has undergone label removal processing. After obtaining a corrected label corresponding to the second type of image dataset, the accuracy determination module is specifically configured to:

[0210] Performing an accuracy test on the prediction model using a preset accuracy test indicator to obtain an accuracy test result;

[0211] In response to the accuracy test result being within a threshold range corresponding to the accuracy test indicator, determining that the correction label corresponding to the second type of image data set is correct; or,

[0212] In response to the accuracy test result not being within a threshold range corresponding to the accuracy test indicator, determining that the correction label corresponding to the second type of image dataset is incorrect, wherein the accuracy test indicator includes at least one of the following:

[0213] Precision index, recall index, F1 score index and intersection-over-union index.

[0214] For the convenience of description, the above devices are described as being divided into various modules according to their functions. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0215] The apparatus of the above embodiment is used to implement the corresponding image processing method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0216] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the program, the image processing method described in any of the above embodiments is implemented.

[0217] Figure 4 A more specific hardware structure diagram of an electronic device provided in this embodiment is shown. The device may include: a processor 401, a memory 402, an input / output interface 403, a communication interface 404, and a bus 405. The processor 401, the memory 402, the input / output interface 403, and the communication interface 404 are communicatively connected to each other within the device via the bus 405.

[0218] The processor 401 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0219] The memory 402 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 402 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 402 and is called and executed by the processor 401.

[0220] The input / output interface 403 is used to connect to input / output modules to implement information input and output. The input / output modules can be configured as components within the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0221] The communication interface 404 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WIFI, Bluetooth, etc.).

[0222] The bus 405 comprises a pathway for transmitting information between various components of the device (eg, the processor 401 , the memory 402 , the input / output interface 403 , and the communication interface 404 ).

[0223] It should be noted that although the above device only shows the processor 401, the memory 402, the input / output interface 403, the communication interface 404, and the bus 405, in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the embodiments of this specification, and does not necessarily include all the components shown in the figure.

[0224] The electronic device of the above embodiment is used to implement the corresponding image processing method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0225] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the image processing method described in any of the above embodiments.

[0226] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0227] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the image processing method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0228] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. Within the scope of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0229] In addition, for simplicity of description and discussion, and in order not to make the embodiment of the application difficult to understand, the known power supply / ground connection with integrated circuit (IC) chip and other components may or may not be shown in the accompanying drawings provided. In addition, the device can be shown in the form of a block diagram to avoid making the embodiment of the application difficult to understand, and this also takes into account the following fact, that is, the details of the embodiment of these block diagram devices are highly dependent on the platform to be implemented in the embodiment of the application (that is, these details should be fully within the scope of understanding of those skilled in the art). When specific details (for example, circuit) are set forth to describe exemplary embodiments of the application, it will be apparent to those skilled in the art that the embodiment of the application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0230] Although the present invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may utilize the embodiments discussed.

[0231] The embodiments of the present application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of this application.

Claims

1. An image processing method, characterized in that: include: Obtain the cluster labels corresponding to the target image data in the initial image dataset at different prediction times and the prediction labels corresponding to different prediction times; Comparing the cluster labels corresponding to a preset number of adjacent prediction times to obtain a first comparison result, and comparing the prediction labels corresponding to a preset number of adjacent prediction times to obtain a second comparison result; In response to the first comparison results both being low loss values ​​and the second comparison results being the same, determining that the label of the target image data is a non-noise label, and combining all target image data labeled with the non-noise label in the initial image dataset as a first type image dataset; In response to the first comparison result being different, and / or the first comparison results both being high loss values, and / or the second comparison result being different, determining that the label of the target image data is a noise label, and combining all target image data labeled with the noise label in the initial image dataset as a second type of image dataset; Using the first type of image dataset to train a pre-built initial model to obtain a prediction model; performing label removal processing on the second type of image dataset, and performing label correction on the second type of image dataset that has undergone label removal processing using the prediction model to obtain a corrected label corresponding to the second type of image dataset; Also includes: counting the number of the second-type image datasets and the number of the initial image datasets, and performing ratio processing on the number of the second-type image datasets and the number of the initial image datasets to obtain a first ratio processing result; Determining the number of noise labels in the second type of image data set provided with the correction labels, and performing ratio processing on the number of noise labels and the number of labels in the initial image data set to obtain a second ratio processing result; In response to the second ratio processing result being less than the first ratio processing result, determining that the correction label corresponding to the second type of image data set is correct; or, In response to the second ratio processing result being greater than or equal to the first ratio processing result, it is determined that the correction label corresponding to the second type of image data set is incorrect.

2. The method according to claim 1, characterized in that The step of performing label correction on the second type of image dataset that has undergone label removal processing by using the prediction model to obtain a corrected label corresponding to the second type of image dataset includes: Performing strong enhancement processing on the second type image dataset that has undergone label removal processing to obtain a second type image dataset that has undergone strong enhancement processing, and performing weak enhancement processing on the second type image dataset that has undergone label removal processing to obtain a second type image dataset that has undergone weak enhancement processing; Inputting the second type of image data set that has undergone strong enhancement processing into the prediction model, outputting a first prediction result through the prediction model, and inputting the second type of image data set that has undergone weak enhancement processing into the prediction model, outputting a second prediction result through the prediction model; constructing a first loss function based on the first prediction result and the second prediction result, performing a minimization process on the first loss function, and training and adjusting the prediction model according to a result of minimizing the first loss function during the minimization process to obtain a trained prediction model; The second type of image data set that has undergone label removal processing is input into the trained prediction model to obtain a correction label corresponding to the second type of image data set.

3. The method according to claim 1, characterized in that The obtaining of cluster labels corresponding to target image data in the initial image dataset at different prediction times includes: Obtaining initial labels of target image data in the initial image dataset; Performing a difference process using the initial label of the target image data and the predicted labels corresponding to different prediction times to obtain loss values ​​corresponding to different prediction times; The k-means clustering algorithm is used to cluster the loss values ​​corresponding to different prediction times, and the cluster labels corresponding to the target image data in the initial image dataset at different prediction times are obtained.

4. The method according to claim 1, wherein The pre-built initial model is trained using the first type of image dataset to obtain a prediction model, including: Obtain initial labels for the first type of image dataset and build an initial model; Inputting the first type of image data set into the initial model, and outputting a training prediction label through the initial model; A second loss function is constructed based on the initial labels of the first type of image data set and the predicted labels for training, and the second loss function is minimized. The initial model is trained and adjusted according to the result of minimizing the second loss function during the minimization process to obtain the prediction model.

5. The method according to claim 1, wherein The pre-built initial model is trained using the first type of image dataset to obtain a prediction model, including: Performing strong enhancement processing on the first type of image dataset to obtain a first type of image dataset that has undergone strong enhancement processing; Obtaining initial labels of the first type of image dataset that has undergone strong enhancement processing and building an initial model; Inputting the first type of image data set that has undergone strong enhancement processing into the initial model, and outputting a training prediction label through the initial model; A third loss function is constructed based on the initial labels of the first type of image data set that has undergone strong enhancement processing and the predicted labels for training, and the third loss function is minimized. The initial model is trained and adjusted according to the result of minimizing the third loss function during the minimization process to obtain the prediction model.

6. The method according to claim 1, characterized in that After performing label correction on the second type of image dataset that has undergone label removal processing using the prediction model to obtain a corrected label corresponding to the second type of image dataset, the method further includes: Performing an accuracy test on the prediction model using a preset accuracy test indicator to obtain an accuracy test result; In response to the accuracy test result being within a threshold range corresponding to the accuracy test indicator, determining that the correction label corresponding to the second type of image data set is correct; or, In response to the accuracy test result not being within a threshold range corresponding to the accuracy test indicator, determining that the correction label corresponding to the second type of image dataset is incorrect, wherein the accuracy test indicator includes at least one of the following: Precision index, recall index, F1 score index and intersection-over-union index.

7. An image processing device, characterized in that: include: An acquisition module is configured to acquire cluster labels corresponding to target image data in the initial image data set at different prediction times and prediction labels corresponding to different prediction times; a comparison module configured to compare cluster labels corresponding to a preset number of adjacent prediction times to obtain a first comparison result, and to compare prediction labels corresponding to a preset number of adjacent prediction times to obtain a second comparison result; a first combining module configured to, in response to the first comparison results both being low loss values ​​and the second comparison results being the same, determine that the label of the target image data is a non-noise label, and combine all target image data labeled with the non-noise label in the initial image dataset as a first type of image dataset; a second combining module configured to, in response to the first comparison result being different, and / or the first comparison results both being high loss values, and / or the second comparison result being different, determine that the label of the target image data is a noise label, and combine all target image data labeled with the noise label in the initial image dataset as a second type of image dataset; a training module configured to train a pre-built initial model using the first type of image dataset to obtain a prediction model; a label correction module configured to perform label removal processing on the second type of image dataset, and perform label correction on the second type of image dataset after the label removal processing using the prediction model to obtain a correction label corresponding to the second type of image dataset; Also configured to: counting the number of the second-type image datasets and the number of the initial image datasets, and performing ratio processing on the number of the second-type image datasets and the number of the initial image datasets to obtain a first ratio processing result; Determining the number of noise labels in the second type of image data set provided with the correction labels, and performing ratio processing on the number of noise labels and the number of labels in the initial image data set to obtain a second ratio processing result; In response to the second ratio processing result being less than the first ratio processing result, determining that a correction label corresponding to the second type of image dataset is correct; or, In response to the second ratio processing result being greater than or equal to the first ratio processing result, it is determined that the correction label corresponding to the second type of image data set is incorrect.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 6.

Images