Data cleaning method, device and storage medium of image dataset

Abstract

The application discloses a data cleaning method and device for an image data set and a storage medium. The method comprises the following steps: performing sampling feature analysis on an obtained to-be-cleaned data set to obtain a feature analysis result; determining a target cleaning strategy from a preset cleaning strategy set according to the feature analysis result; arranging the target cleaning strategy to obtain a cleaning task; and executing the cleaning task on the to-be-cleaned data set to obtain a data cleaning result. The above scheme can improve the data cleaning efficiency and cleaning effect.

Description

Technical Field

[0001] This application relates to the field of data cleaning technology, and in particular to a data cleaning method, apparatus, and storage medium for an image dataset. Background Technology

[0002] The performance of deep learning models (especially convolutional neural networks, Vision Transformers, etc.) is highly dependent on the quality and scale of the training data. High-quality image datasets are the foundation for models to achieve good generalization ability, robustness, and detection accuracy.

[0003] However, in actual engineering, the original image dataset often has a variety of quality problems, such as invalid images, low-quality images, duplicate images, class imbalance, label noise, etc., all of which will seriously affect the model training effect.

[0004] Although various image cleaning techniques exist, current image cleaning technologies often face problems such as fragmented cleaning tools, reliance on manual setting of cleaning parameters, and a lack of verification of cleaning effects, resulting in low data cleaning efficiency and poor cleaning results. Summary of the Invention

[0005] This application provides at least one method, apparatus, device, and computer-readable storage medium for cleaning image datasets.

[0006] The first aspect of this application provides a data cleaning method for an image dataset, comprising: performing sampling feature analysis on the acquired dataset to be cleaned to obtain feature analysis results; determining a target cleaning strategy in a preset cleaning strategy set based on the feature analysis results; arranging the target cleaning strategy to obtain a cleaning task; and executing the cleaning task on the dataset to be cleaned to obtain a data cleaning result.

[0007] In one embodiment, the step of performing sampling feature analysis on the acquired dataset to be cleaned to obtain feature analysis results includes: sampling the dataset to be cleaned to obtain sampled data; performing feature analysis on the sampled data to obtain data features of the sampled data; and performing anomaly index analysis on the sampled data based on the data features to obtain the feature analysis results.

[0008] In one embodiment, the step of performing anomaly index analysis on the sampled data based on the data features to obtain the feature analysis result includes: determining abnormal data in the sampled data based on the data features, wherein the abnormal data includes at least one of low-quality data and duplicate data; determining anomaly index corresponding to each type of abnormal data based on the number of anomalies in each type of abnormal data and the number of samples in the sampled data; and comparing the anomaly index with a preset anomaly threshold to obtain the feature analysis result.

[0009] In one embodiment, the cleaning strategy set includes at least a quality cleaning strategy, a deduplication cleaning strategy, and a classification cleaning strategy. The step of determining a target cleaning strategy from the preset cleaning strategy set based on the feature analysis results includes: if the feature analysis results indicate that the data quality of the dataset to be cleaned does not meet preset data quality conditions, then the quality cleaning strategy and the classification cleaning strategy are determined as the target cleaning strategy; if the feature analysis results indicate that the data duplication rate of the dataset to be cleaned does not meet preset data duplication conditions, then the deduplication cleaning strategy and the classification cleaning strategy are determined as the target cleaning strategy; if the feature analysis results indicate that the data quality of the dataset to be cleaned meets the data quality conditions, and the data duplication rate of the dataset to be cleaned meets the data duplication conditions, then the classification cleaning strategy is determined as the target cleaning strategy.

[0010] In one embodiment, orchestrating the target cleaning strategies to obtain a cleaning task includes: obtaining the strategy complexity of each target cleaning strategy; and orchestrating the target cleaning strategies in ascending order of strategy complexity to obtain the cleaning task.

[0011] In one embodiment, the cleaning task includes a label cleaning task, or at least one of a low-quality cleaning task and a deduplication cleaning task, plus the label cleaning task. Performing the cleaning task on the dataset to be cleaned to obtain a data cleaning result includes: performing the label cleaning task on the dataset to be cleaned, or performing at least one of the low-quality cleaning task and the deduplication cleaning task, plus the label cleaning task, on the dataset to be cleaned to obtain the data cleaning result; wherein the low-quality cleaning task includes: determining low-quality data in the dataset to be cleaned based on image attributes; filtering the low-quality data; and / or, the deduplication cleaning task includes... The process includes: determining duplicate data in the dataset to be cleaned based on image similarity among the data to be cleaned; performing deduplication on the duplicate data; and / or, the label cleaning task includes: classifying the data to be cleaned in the dataset to obtain classification labels and classification confidence scores; selecting target data from the data to be cleaned based on the classification confidence scores to train a surrogate classifier, obtaining the surrogate classifier; inputting the data to be cleaned into the surrogate classifier to obtain predicted labels for the data to be cleaned; determining noise labels based on the consistency between the classification labels and predicted labels of the same data to be cleaned; and performing correction processing on the noise labels.

[0012] In one embodiment, classifying the data to be cleaned in the dataset to be cleaned to obtain classification labels and classification confidence scores includes: inputting the data to be cleaned into a pre-trained scene classifier and a pre-trained target classifier respectively to obtain scene prediction labels output by the scene classifier and target prediction labels output by the target classifier; determining the classification label based on the scene prediction label and the target prediction label of the same data to be cleaned; and determining the classification confidence score based on the scene classification confidence score corresponding to the scene prediction label and the target detection confidence score corresponding to the target prediction label.

[0013] In one embodiment, the duplicate data includes first duplicate data and second duplicate data. The step of determining duplicate data in the dataset to be cleaned based on image similarity between each piece of data to be cleaned, and performing deduplication on the duplicate data, includes: hashing each piece of data to be cleaned to obtain a hash value; determining the hash Hamming distance between every two pieces of data to be cleaned based on the hash value; identifying the data to be cleaned with a hash Hamming distance less than a preset Hamming distance threshold as the first duplicate data; obtaining the remaining data to be cleaned excluding the first duplicate data; determining the pixel similarity between every two remaining pieces of data to be cleaned based on the pixel values ​​of the remaining data; identifying the remaining data to be cleaned with a pixel similarity greater than a preset pixel similarity threshold as the second duplicate data; and performing deduplication on the first duplicate data and the second duplicate data.

[0014] In one embodiment, the method further includes: performing a cleaning effect verification process on the low-quality cleaning task to obtain a first verification result; if the first verification result does not meet a preset first verification condition, adjusting the low-quality cleaning parameters of the low-quality cleaning task according to the first verification result; and / or performing a cleaning effect verification process on the deduplication cleaning task to obtain a second verification result; if the second verification result does not meet a preset second verification condition, adjusting the deduplication cleaning parameters of the deduplication cleaning task according to the second verification result.

[0015] In one embodiment, the step of performing cleaning effect verification processing on the low-quality cleaning task to obtain a first verification result includes: performing negative data injection processing on the pre-cleaning data corresponding to the low-quality cleaning task to obtain a low-quality cleaning verification dataset; and performing the low-quality cleaning task and recall calculation on the low-quality cleaning verification dataset to obtain the first verification result.

[0016] In one embodiment, the step of performing cleaning effect verification processing on the deduplication cleaning task to obtain a second verification result includes: performing negative data injection processing on the pre-cleaning data corresponding to the deduplication cleaning task to obtain a deduplication cleaning verification dataset; and performing the deduplication cleaning task and recall calculation on the deduplication cleaning verification dataset to obtain the second verification result.

[0017] In one embodiment, the step of verifying the cleaning effect of the deduplication cleaning task to obtain a second verification result includes: sampling the cleaned data corresponding to the deduplication cleaning task to obtain deduplication cleaning sample data; obtaining a similarity matrix between every two deduplication cleaning sample data; determining deduplication cleaning similar data whose similarity to a specified region is greater than a preset feature similarity threshold based on the similarity of a specified region in the similarity matrix; determining the deduplication cleaning similarity ratio based on the number of deduplication cleaning sample data and the number of deduplication cleaning similar data; and determining the second verification result based on the comparison between the deduplication cleaning similarity ratio and a preset deduplication cleaning similarity ratio threshold.

[0018] In one embodiment, after performing the cleaning task on the dataset to be cleaned and obtaining the data cleaning result, the method further includes: performing quality assessment processing on the cleaned dataset in the data cleaning result to obtain a quality analysis report; and determining whether to perform data optimization processing on the cleaned dataset based on the quality analysis report.

[0019] In one embodiment, determining whether to perform data optimization processing on the cleaned dataset based on the quality analysis report includes: determining the quality score of each cleaned data in the cleaned dataset based on the quality analysis report; determining the data to be optimized in each cleaned data based on the quality score; and performing image enhancement optimization processing on the data to be optimized.

[0020] In one embodiment, determining whether to perform data optimization processing on the cleaned dataset based on the quality analysis report includes: determining the number of samples corresponding to each data type in the cleaned dataset based on the quality analysis report; and performing data supplementation optimization processing on data types whose sample number is less than a preset sample number threshold.

[0021] In one embodiment, after performing the cleaning task on the dataset to be cleaned and obtaining the data cleaning result, the method further includes: verifying the cleaning effectiveness of the cleaned dataset in the data cleaning result according to a preset model training task to obtain the cleaning effectiveness of the cleaned dataset; if the cleaning effectiveness does not meet the preset cleaning effectiveness conditions, then adjusting the cleaning parameters of the target cleaning strategy.

[0022] In one embodiment, the method further includes: identifying difficult samples in the cleaned dataset when performing the preset model training task based on a preset confidence threshold and a loss threshold; adjusting the cleaning parameters of the target cleaning strategy based on the image features of the difficult samples; and marking and retaining key difficult samples containing key targets among the difficult samples.

[0023] A second aspect of this application provides a data cleaning apparatus for an image dataset, comprising: a data analysis module for performing sampling feature analysis on the acquired dataset to be cleaned to obtain feature analysis results; a strategy determination module for determining a target cleaning strategy from a preset cleaning strategy set based on the feature analysis results; a strategy orchestration module for orchestrating the target cleaning strategy to obtain a cleaning task; and a cleaning execution module for executing the cleaning task on the dataset to be cleaned to obtain a data cleaning result.

[0024] A third aspect of this application provides an electronic device, including a memory and a processor, wherein the processor is configured to execute program instructions stored in the memory to implement the data cleaning method for the aforementioned image dataset.

[0025] The fourth aspect of this application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the data cleaning method for the image dataset described above.

[0026] The technical solution of this application involves sampling and feature analysis of the acquired dataset to be cleaned to obtain feature analysis results. These results typically characterize any anomalies or defects present in the dataset. Based on these results, a suitable target cleaning strategy can be determined from a pre-defined set of cleaning strategies. The selected target cleaning strategies are then arranged to form a cleaning task. By executing the cleaning task on the data to be cleaned, the cleaned dataset is obtained. This approach significantly improves data cleaning efficiency by enabling the selection and arrangement of appropriate cleaning strategies based on the actual defects of the data.

[0027] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0028] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0029] Figure 1 This is a flowchart illustrating an exemplary embodiment of the data cleaning method for the image dataset of this application; Figure 2 This is a schematic diagram of an exemplary data cleaning system in the data cleaning method for image datasets of this application; Figure 3 This is a block diagram illustrating a data cleaning apparatus for an image dataset, as shown in an exemplary embodiment of this application. Figure 4 This is a schematic diagram of the structure of an embodiment of the electronic device of this application; Figure 5 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0030] The solution of this embodiment will now be described in detail with reference to the accompanying drawings.

[0031] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0032] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0033] For ease of understanding, one of the applicable scenarios of this application is illustrated by example.

[0034] In practical engineering for training neural network models, the original image dataset often has one or more quality problems, which seriously affect the training effect of the model.

[0035] For example, the image dataset may contain invalid images that lack effective visual information (such as images with extremely low information entropy (solid color / near-solid color), extreme size ratios, or distorted images), low-quality images (such as blurry images, abnormal brightness / contrast, or images with excessive noise), identical or similar duplicate images, or an unbalanced distribution of image categories.

[0036] Although there are various image cleaning techniques available, which can remove abnormal samples from the original dataset through automated or semi-automated methods.

[0037] However, the inventors of this application have discovered that in the image cleaning process, not only are the cleaning tools fragmented, but it also requires manual arrangement of tools and setting of cleaning parameters, and the cleaning results are not verified, resulting in low data cleaning efficiency and poor cleaning effect.

[0038] Please see Figure 1 , Figure 1 This is a flowchart illustrating an exemplary embodiment of the data cleaning method for the image dataset of this application. Specifically, it may include the following steps: Step S110: Perform sampling feature analysis on the obtained dataset to be cleaned to obtain feature analysis results.

[0039] Typically, the dataset to be cleaned includes multiple data samples.

[0040] The dataset to be cleaned may be obtained from image acquisition devices, and / or through image synthesis, image generation, or other technical means, and / or through the Internet; there are no restrictions here.

[0041] Sampling feature analysis refers to the analysis of data features of a sampled portion of the dataset to be cleaned. This analysis may include, but is not limited to, analyzing the data features of the sampled data (e.g., image attributes, data quality) and / or analyzing the data features between multiple sampled data sets (e.g., data similarity). This can, to a certain extent, characterize the feature analysis results of the dataset to be cleaned.

[0042] For example, feature analysis results can characterize whether there are any abnormal defects in the data to be cleaned (single or multiple data) within the dataset to be cleaned. If defects are found, they need to be cleaned.

[0043] It is understood that, in order to facilitate the explanation of the data cleaning method of this application, this application mainly takes the application scenario where the data in the dataset to be cleaned has defects as an example.

[0044] Step S120: Determine the target cleaning strategy from the preset cleaning strategy set based on the feature analysis results.

[0045] The cleaning strategy set may include at least one data cleaning strategy, such as invalid data removal (abnormal data removal), quality filtering, data deduplication, noise and blemish repair, format and size standardization, data augmentation, etc., without limitation.

[0046] It should be noted that invalid data and abnormal data (such as corrupted files that cannot be decoded, sensitive images, solid color images, images that are not the target content, etc.) are also considered low-quality data according to certain evaluation criteria. Therefore, for the convenience of subsequent examples, invalid data, abnormal data, and low-quality data can be collectively referred to as low-quality data in the embodiments of this application.

[0047] In this application, based on the defects of the dataset reflected by the feature analysis results, an appropriate target cleaning strategy can be selected from the cleaning strategy set. In some application scenarios, unnecessary cleaning steps can be skipped, so that the defects of the dataset can be solved by the target cleaning strategy. Instead of using a fixed and rigid cleaning method as in traditional methods, which would have to execute all the preset cleaning steps and thus waste computing resources.

[0048] Step S130: Arrange the target cleaning strategy to obtain the cleaning task.

[0049] Based on the steps described above, one or more target cleaning strategies can be selected, depending on the feature analysis results, and no limit is imposed here.

[0050] If there is only one target cleaning strategy, a cleaning task can be generated based on the target cleaning strategy, and the cleaning task can be executed to complete the cleaning process of the data to be cleaned and obtain the data cleaning result.

[0051] If there are multiple target cleaning strategies, they need to be orchestrated to determine the execution order of each strategy, thus obtaining cleaning tasks. These tasks are then executed to complete the cleaning of the data, yielding the cleaned data results.

[0052] For example, the orchestration processing method in this application may include, but is not limited to, one or more of the following: orchestration based on the preset priority of each cleaning strategy, orchestration based on the algorithm complexity of each cleaning strategy, and orchestration based on the data throughput of each cleaning strategy, etc., which will not be elaborated here.

[0053] By orchestrating the cleaning strategies for each target, the execution order of each cleaning strategy and the computing resources to be allocated in the cleaning task can be determined, which will not be elaborated here.

[0054] Optionally, it can also be determined whether the size N of the dataset to be cleaned or the data to be cleaned is greater than a preset size threshold. (e.g., 1 million or other customizable values). If it is ( If this is the case, then distributed parallel processing can be enabled, and the cleaning strategies for each target can be orchestrated for parallel execution.

[0055] The cleaning tasks output after strategy orchestration may include, but are not limited to: a list of phases for which the strategy is enabled, initial values ​​of cleaning intensity for each phase, and a computing resource allocation scheme.

[0056] Step S140: Perform a cleaning task on the dataset to be cleaned to obtain the data cleaning results.

[0057] Based on the steps described above, after obtaining the cleaning task, the cleaning task can be executed on the data to be cleaned to obtain the output data cleaning results.

[0058] The data cleaning result can include whether the cleaning was successful or failed.

[0059] Understandably, if the data cleaning result indicates successful cleaning, the cleaned dataset can be used entirely or partially as a training set for model training (further data processing is also possible, but will not be elaborated here). Alternatively, a portion of the cleaned dataset can be selected as a test set, but this will also not be elaborated here.

[0060] For example, it can be applied to, but is not limited to: object detection model training (training object detection models such as YOLO and Faster R-CNN), image classification model training (training classification models such as ResNet and EfficientNet), semantic segmentation model training (training segmentation models such as U-Net and DeepLab), and multimodal model training (training multimodal models such as CLIP and BLIP), which will not be elaborated here.

[0061] If the data cleaning result indicates that the cleaning has failed, the cleaning process can be performed on the dataset again (either by performing the same cleaning task or by re-determining and orchestrating the target cleaning strategy, which is not limited here); or, log files such as cleaning reports indicating the cleaning failure can be generated for subsequent auditing, which will not be elaborated here.

[0062] As can be seen, this application obtains feature analysis results by performing sampling feature analysis on the acquired dataset to be cleaned. The feature analysis results can typically characterize the abnormal defects existing in the dataset to be cleaned. Therefore, based on the feature analysis results, a suitable target cleaning strategy can be determined from a preset cleaning strategy set. Then, the selected target cleaning strategies are arranged to obtain the cleaning task. By executing the cleaning task on the data to be cleaned, the data cleaning result, i.e., the cleaned dataset to be cleaned, can be obtained. This achieves the goal of selecting and arranging suitable cleaning strategies based on the actual defects of the data to be cleaned, significantly improving data cleaning efficiency.

[0063] Based on the above embodiments, this embodiment should also explain that after performing step S140, the obtained data cleaning results can be evaluated and adjusted, for example, it can include at least the following steps S150 to S160: Step S150: Perform quality assessment on the cleaned dataset in the data cleaning results to obtain a quality analysis report.

[0064] In conjunction with the steps outlined above, the data cleaning result can also include the cleaned dataset after cleaning. Therefore, a quality analysis report can be obtained by performing a quality assessment on the cleaned dataset. This quality analysis report characterizes the cleaning effect on the dataset to be cleaned.

[0065] The quality assessment method can be set as needed according to the actual application scenario. For example, it can assess the cleaning rate, the quality of the cleaned image, and the distribution of data categories after cleaning. There are no restrictions here.

[0066] Step S160: Determine whether to perform data optimization processing on the cleaned dataset based on the quality analysis report.

[0067] Based on the aforementioned steps, if the quality analysis report indicates that the evaluation indicators obtained from the above assessment do not meet the corresponding preset quality assessment conditions, then data optimization processing can be performed on the cleaned dataset; if the quality analysis report indicates that the evaluation indicators obtained from the above assessment meet the corresponding preset quality assessment conditions, then data optimization processing on the cleaned dataset is not necessary.

[0068] The data optimization methods can be set as needed according to the actual application scenario, and are not limited here. Different data optimization methods can be used for different types of evaluation indicators. For example, for data with poor image quality after cleaning, image enhancement optimization can be performed; for data with uneven distribution of data categories after cleaning, image quantity can be increased, etc.

[0069] As can be seen, after obtaining the data cleaning results, this application can also choose to perform quality assessment processing on the cleaned dataset in the data cleaning results to obtain a quality analysis report; and determine whether to perform data optimization processing on the cleaned dataset based on the quality analysis report, thus completing the closed-loop assessment and data optimization of the data cleaning task.

[0070] Based on the above embodiments, this embodiment describes the method in step S110.

[0071] Specifically, the method for performing sampling feature analysis on the acquired dataset to be cleaned and obtaining the feature analysis results in this embodiment includes at least the following steps S210 to S230: Step S210: Sample the dataset to be cleaned to obtain sampled data.

[0072] Step S220: Perform feature analysis on the sampled data to obtain the data features of the sampled data.

[0073] Step S230: Perform anomaly index analysis on the sampled data based on data characteristics to obtain feature analysis results.

[0074] As described in conjunction with the foregoing embodiments, the dataset used for model training typically includes massive amounts of data. Therefore, in this application, a portion of the data can be sampled from the dataset to be cleaned to characterize the data features of the entire dataset through data sampling.

[0075] It should be noted that the sampling method of this application can be set as needed according to the actual application scenario, and may include, but is not limited to: simple random sampling, systematic sampling, stratified sampling, multi-stage sampling, etc., which are not limited here.

[0076] For example, random sampling from the dataset to be cleaned A single image, typical sampling ratio It can be set to 1%~5% (or can be set as needed according to the actual application scenario), that is .

[0077] It should also be noted that the dataset to be cleaned The data may be in different formats. This represents the total number of images. Each image's data... (N≥k≥1) may have different formats (such as BMP, TIFF, WEBP, PNG, JPG, etc.), different sizes, different color spaces, etc., which are not limited here.

[0078] Therefore, before performing data sampling processing, format unification can also be performed. For example, all image data can be converted to a standard format (such as PNG or JPG, which can be set as needed according to the actual application scenario): ,in The target format is specified. For images with inconsistent color spaces (such as CMYK, grayscale, etc.), they can be converted to a unified color space (such as RGB), which is not limited here.

[0079] Optionally, after the format conversion is completed, all or some images in the dataset to be cleaned, or all or some images after format conversion, can be format verified to obtain the format verification results of the image data (such as the readability and / or integrity of the image data).

[0080] Therefore, based on the format verification results, corrupted images that cannot be decoded normally can be removed, resulting in a uniformly formatted dataset to be cleaned. Then, the data to be cleaned is subjected to sampling feature analysis and processing.

[0081] Based on the above embodiments, this embodiment describes the method in step S230.

[0082] Specifically, the method for performing anomaly index analysis on sampled data based on data characteristics to obtain feature analysis results in this embodiment includes at least the following steps S310 to S330: Step S310: Determine abnormal data in the sampled data based on data characteristics. Abnormal data includes at least one of low-quality data and duplicate data.

[0083] For details regarding low-quality data, duplicate data, and other possible abnormal data, please refer to the descriptions in other embodiments of this application, which will not be repeated here.

[0084] Step S320: Determine the abnormal indicators corresponding to each type of abnormal data based on the number of abnormalities in each type of abnormal data and the number of samples in the sampled data.

[0085] For example, the anomaly indicator (low quality indicator) corresponding to low quality data can be determined based on the number of low quality data (i.e., the number of anomalies) and the number of sampled data (i.e., the number of samples).

[0086] For example, in some implementations, a blurred image can be identified as a low-quality image, and thus the low-quality index can be determined by the ratio between the number of blurred images and the number of samples (which may be called the blurred image ratio).

[0087] This can be achieved by, but is not limited to, calculating the Laplacian variance of each sampled image based on the Laplacian operator, and identifying sampled images with a Laplacian variance below a blur threshold as blurred images. From this, the proportion of blurred images among the sampled images can be calculated. .

[0088] In addition, depending on the actual application scenario, operators such as Sobel can be selected to calculate the variance, or frequency domain analysis methods can be used to characterize the degree of blur in the image. This will not be elaborated here.

[0089] For example, in some implementations, noisy images can be identified as low-quality images, and thus the low-quality index can be determined by the ratio between the number of noisy images and the number of samples (which may be called the noisy image ratio).

[0090] This can be achieved by methods including, but not limited to, high-frequency energy analysis or blind noise estimation to detect image noise values, and identifying sampled images with noise values ​​exceeding a preset noise threshold as noisy images. From this, the proportion of noisy images to the total sampled images can be calculated.

[0091] For example, in some implementations, images with abnormal brightness can be identified as low-quality images. Therefore, the low-quality index can be determined by the ratio between the number of images with abnormal brightness and the number of samples (which can be called the ratio of images with abnormal brightness).

[0092] This can be achieved by, but is not limited to, calculating the average brightness of each sampled image. Images with a mean brightness value higher than a preset overexposure threshold or lower than a preset underexposure threshold are identified as images with abnormal brightness. Therefore, the proportion of images with abnormal brightness among the sampled images can be calculated. .

[0093] In addition, it is also possible to select by calculating the brightness standard deviation of each sampled image. The sampled images with a brightness standard deviation lower than the preset contrast threshold are identified as images with abnormal contrast, and their corresponding abnormal proportions are determined in the same way.

[0094] It should be noted that low-quality images can be identified using one or more low-quality indicators. For example, a sampled image can be classified as a low-quality image if it meets any of the anomaly conditions in the examples above, or it may be classified as a low-quality image if it meets at least two of the anomaly conditions in the examples above. This is not limited here.

[0095] Another example is that the abnormal indicator (duplication indicator) corresponding to duplicate data can be determined based on the number of duplicate data (i.e., the number of duplicates) and the number of sampled data (i.e., the number of samples).

[0096] For example, in some implementations, duplicate images can be identified by calculating the hash collision rate between images, feature similarity, etc., and the proportion of duplicate images in the sampled images (which can be called the duplication rate) can be calculated to determine the duplication index. .

[0097] Step S330: Compare the abnormal indicators with the preset abnormal thresholds to obtain the feature analysis results.

[0098] Among them, the preset abnormal threshold can be used to measure the magnitude of abnormal indicators.

[0099] For example, if an abnormal indicator is greater than or equal to the corresponding preset abnormal threshold, it indicates that there is a large number of abnormal data corresponding to the abnormal indicator, and the abnormal defect is determined to be valid (and corresponding cleaning processing is required).

[0100] If the abnormal indicator is less than the corresponding preset abnormal threshold, it indicates that the number of abnormal data corresponding to the abnormal indicator is small, and the abnormal defect is determined to be invalid (no corresponding cleaning process is required).

[0101] There can be one or more preset abnormal thresholds. Different abnormal indicators can correspond to the same or different preset abnormal thresholds, which is not limited here.

[0102] Specifically, in one possible implementation, the repetition rate The feature analysis results are obtained by comparing the results with the corresponding preset anomaly threshold (which can be set to 0.01 or other values).

[0103] If the repetition rate in the feature analysis results A value greater than 0.01 indicates the presence of numerous duplicate images (a repetition defect), thus requiring further deduplication processing of the dataset to be cleaned. Conversely, a higher repetition rate indicates a lower repetition rate. If the value is less than or equal to 0.01, then the re-cleaning process can be skipped.

[0104] Similarly, in another possible implementation, the blurring image ratio The feature analysis results are obtained by comparing the results with the corresponding preset anomaly threshold (which can be set to 0.5 or other values).

[0105] If the proportion of blurred images in the feature analysis results A value greater than 0.5 indicates the presence of many blurry images (indicating low-quality defects), thus requiring subsequent quality filtering of the dataset to be cleaned.

[0106] Based on the above embodiments, this embodiment describes the method in step S120.

[0107] The cleaning strategy set includes at least a quality cleaning strategy, a deduplication cleaning strategy, and a classification cleaning strategy.

[0108] It should be noted that each cleaning strategy in the cleaning strategy set is used to clean image data with corresponding defects, and the specific cleaning strategy can be set as needed according to the actual application scenario.

[0109] For example, quality cleaning strategies are used to clean low-quality images, deduplication cleaning strategies are used to clean duplicate images, and classification cleaning strategies are used to clean images with low classification accuracy (low confidence).

[0110] Specifically, the method for determining a target cleaning strategy from a preset cleaning strategy set based on feature analysis results in this embodiment includes at least the following steps S410 to S430: Step S410: If the feature analysis results indicate that the data quality of the dataset to be cleaned does not meet the preset data quality conditions, then the quality cleaning strategy and the classification cleaning strategy are determined as the target cleaning strategy.

[0111] For example, if the data quality of the data to be cleaned does not meet the preset data quality conditions (e.g., the proportion of low-quality images is greater than the corresponding preset abnormal threshold), a quality cleaning strategy can be selected to be executed.

[0112] There may be one or more reasons for low data quality, so the quality cleaning strategy may also include one or more quality cleaning sub-strategies. For example, it may include sub-strategies for cleaning invalid images, sub-strategies for cleaning blurry images, sub-strategies for cleaning images with abnormal brightness, sub-strategies for cleaning noisy images, etc., without limitation here.

[0113] Specifically, in the sub-strategy of cleaning invalid images, images that do not have effective visual information can be quickly removed. Because of its low computational complexity, its preprocessing can significantly reduce the amount of data processing required by subsequent high-complexity tools.

[0114] For example, sub-strategies for cleaning invalid images may include, but are not limited to: (1) Information entropy filtering Calculate each image Information entropy Its mathematical expression can be:

[0115] in grayscale The probability of occurrence This represents the total number of gray levels (typically 256). Images with extremely low information entropy (near-pure colors) do not possess effective visual information.

[0116] Therefore, the filtering conditions can be: ,in, This is the lower limit threshold for information entropy, typically ranging from 3.0 to 5.0 bits, but other values ​​can also be set as needed.

[0117] (2) Extreme size ratio filtration Calculate the aspect ratio of each image. .

[0118] Filtering conditions can be: or .in and These are the preset lower and upper limits for aspect ratio, with typical values ​​being [values ​​to be filled in]. Other values ​​can also be set as needed.

[0119] (3) Laplacian variance filtering Calculate the Laplacian response variance for each image: .

[0120] Images with extremely low variance are typically solid colors or pixelated images. Therefore, the filtering conditions can be: .in, This is the Laplacian variance threshold (ambiguity threshold), typically ranging from 100 to 300, but other values ​​can be set as needed.

[0121] Optionally, invalid images can be determined and filtered out based on one or more of the above sub-strategies. Furthermore, the execution order of the above sub-strategies can be executed in a certain order, a random order, or concurrently, etc., which is not limited here.

[0122] Specifically, in some implementation methods, the above three filtering conditions can be combined to determine invalid images, resulting in the following set of invalid images:

[0123] Furthermore, the dataset to be cleaned can be cleaned by removing invalid images from the dataset to be cleaned (removing invalid images from the dataset to be cleaned), resulting in a cleaned dataset. .

[0124] It is understandable that the cleaned dataset can be directly output as the data cleaning result, or other cleaning strategies can be applied to it; this is not limited here.

[0125] For example, sub-strategies for cleaning low-quality images may include, but are not limited to: (1) Sharpness detection Image sharpness can be detected using methods such as the Laplacian variance Sobel operator or frequency domain analysis.

[0126] Among them, a sharpness score can be defined. The filtering conditions can be: .

[0127] in, The sharpness threshold can be a preset fixed value or can be adjusted using a filter intensity parameter. The adjustment can be expressed mathematically as follows:

[0128] in A statistical value (e.g., mean) that scores the sharpness of all or part of the image data in the current dataset. This is the preset adjustment coefficient.

[0129] It should be noted that the method for calculating the sharpness of low-quality images in this embodiment may be the same as or different from the method for determining the Laplacian variance when analyzing invalid data in the previous embodiments, and no limitation is made here. Furthermore, the sharpness threshold used in this embodiment may be the same as or different from the blur threshold used in the previous embodiments.

[0130] Optionally, It can be greater than .For example The value can be between 400 and 500. The value can be between 100 and 300. Therefore, images can be distinguished into low-quality images and invalid images by using two thresholds, and two-stage filtering processing (invalid image filtering and low-quality image filtering) can be performed.

[0131] (2) Detection of abnormal brightness / contrast Calculate the average brightness of the image and standard deviation The criteria for determining overexposure are: The criteria for determining underexposure are: The criteria for determining insufficient contrast are: .

[0132] Therefore, images that are overexposed, underexposed, or lack contrast can be identified as low-quality images and then cleaned and filtered.

[0133] (3) Noise detection Image noise levels can be detected using methods such as high-frequency energy analysis or blind noise estimation. An image noise score can be defined. The filtering conditions are: .

[0134] Therefore, the image noise score can be greater than the image noise threshold. The image was identified as a low-quality image and was cleaned and filtered.

[0135] (4) Judgment of low-quality images It should be noted that one or more of the above image attributes (such as blur, sharpness, brightness, contrast, image noise, information entropy, etc.) can be used to judge low-quality images.

[0136] In one possible implementation, the above image attribute detections can be combined, i.e., the low-quality image set can be:

[0137]

[0138] Therefore, after removing the low-quality images from the examples above, we can obtain the dataset to be cleaned after cleaning the low-quality images: .

[0139] Then, you can... Implement a classification and cleaning strategy: The data to be cleaned is classified and processed to obtain... The classification labels and classification confidence scores of each data set to be cleaned are provided. Data sets with higher classification confidence scores can be selected for training the surrogate classifier. .

[0140] Furthermore, Each piece of data to be cleaned is input into the proxy classifier. This will give you the output of the proxy classifier. The predicted labels corresponding to each piece of data to be cleaned.

[0141] Therefore, the initial classification label can be used as the original label, and loss calculation can be performed with the predicted label corresponding to the same data. Then, the presence of label noise in the data can be determined by combining the preset loss threshold, which determines whether label correction should be performed subsequently.

[0142] Step S420: If the feature analysis results indicate that the data duplication rate of the dataset to be cleaned does not meet the preset data duplication conditions, then the deduplication cleaning strategy and the classification cleaning strategy are determined as the target cleaning strategies.

[0143] Referring to the foregoing embodiments, if the data duplication rate of the data to be cleaned does not meet the preset data duplication conditions (such as the data duplication rate being greater than the corresponding preset abnormal threshold), then a deduplication cleaning strategy can be selected to be executed.

[0144] For example, the deduplication and cleaning strategy of this application may similarly include one or more deduplication and cleaning sub-strategies, which can be set as needed according to the actual application scenario, and are not limited here. For example, the deduplication and cleaning sub-strategies of this application may include: image basic deduplication sub-strategy, image feature deduplication sub-strategy, etc.

[0145] Among them, the basic image deduplication sub-strategy may include: fast deduplication based on hash calculation and Hamming distance and / or cosine similarity, cleaning up completely identical or nearly identical images.

[0146] Among them, a deduplication strategy based on hash calculation and Hamming distance and a deduplication strategy based on cosine similarity can be used simultaneously to achieve two-level deduplication.

[0147] For example, duplicate data may include first duplicate data and second duplicate data, and the steps of the image-based deduplication sub-strategy may include at least: Each piece of data to be cleaned is hashed to obtain a hash value; the hash Hamming distance between each pair of data to be cleaned is determined based on the hash value; data to be cleaned whose hash Hamming distance is less than a preset Hamming distance threshold is identified as the first duplicate data. Obtain the remaining data to be cleaned from each set of data to be cleaned, excluding the first duplicate data; determine the pixel similarity between each pair of remaining data to be cleaned based on the pixel values ​​of the remaining data to be cleaned; identify the remaining data to be cleaned with a pixel similarity greater than a preset pixel similarity threshold as the second duplicate data. The first and second duplicate data are deduplicated.

[0148] Specifically, (1) Perceptual hash calculation: For each image The mathematical expression for calculating a perceptual hash (pHash) can be:

[0149] Hamming distance for fast deduplication: Calculate the image pair formed by every two data points to be cleaned. The hash Hamming distance, mathematically expressed as:

[0150] The criteria for deduplication can be set as follows: ,in This is the Hamming distance threshold, typically ranging from 5 to 15.

[0151] FAISS accelerates nearest neighbor retrieval: For large-scale datasets, the FAISS library can be used for efficient nearest neighbor retrieval, avoiding... A full pairwise comparison is performed. An index can be built on all hash vectors, and for each vector, the Hamming distance can be queried to be less than [value missing]. . the neighbor.

[0152] Therefore, when the Hamming distance between the hash values ​​of two images is less than the Hamming distance threshold, the two images can be determined to be similar or identical (first duplicate data), and deduplication processing is required.

[0153] (2) Cosine similarity-assisted deduplication For image pairs that cannot be determined by hash deduplication (all or part of the image pair consisting of the remaining data to be cleaned, for example, the Hamming distance is in the threshold boundary region: the Hamming distance can be equal to the Hamming distance threshold and / or the difference between the Hamming distance and the Hamming distance threshold is less than the preset boundary threshold), pixel-level cosine similarity can be further calculated, and its mathematical expression can be:

[0154] in, For image The flattened pixel vector. and That is, an image and images Flattened pixel vector The criteria for deduplication can be set as follows: That is, the pixel similarity is greater than the pixel similarity threshold. The two images i and j are duplicate images (identical or similar), so it is necessary to remove duplicates from the two images (the second duplicate data) and keep only one of them.

[0155] This can involve randomly selecting one image or selecting one image from a set of predefined deduplication rules. For example, predefined deduplication rules may include, but are not limited to, selecting images based on image attributes or selecting images based on the order of data traversal; no specific rules are specified here.

[0156] For example, in scenarios where image attributes are selected for retention, the following may be included but not limited to: selecting for retention based on image creation time (e.g., retaining images with shorter creation times, i.e., retaining the newest images), selecting for retention based on image information entropy (e.g., retaining images with higher information entropy), and selecting for retention based on image data size (e.g., retaining images with larger data sizes), etc., which will not be elaborated here.

[0157] After performing one or more basic deduplication steps as shown in the examples above, a candidate set of deduplications can be obtained. And record the "one-to-many" mapping relationship.

[0158] In addition, image feature deduplication strategies may include, but are not limited to: semantic-level deduplication based on deep feature extraction, which can detect semantically repetitive images such as similar viewpoints, slight cropping, and similar scenes.

[0159] Deep feature extraction: Each image is extracted using models such as CNN (e.g., ResNet) or CLIP. The image feature vector can be mathematically expressed as:

[0160] in The dimensions of image features (e.g., ResNet50 can output 2048 dimensions, CLIP can output 512 dimensions or 768 dimensions).

[0161] Alternatively, CLIP can be replaced with other vision-language pre-trained models (such as ALIGN, FILIP) or pure vision models (such as DINO, MAE), etc., without limitation here.

[0162] Cosine similarity calculation: Calculate image pairs The feature cosine similarity (feature similarity) can be mathematically expressed as:

[0163] Semantic deduplication determination: The criteria for deduplication can be set as follows: .

[0164] in, This is the feature similarity threshold, typically ranging from 0.85 to 0.95, but other values ​​can also be set as needed.

[0165] Therefore, two images with a feature similarity greater than the feature similarity threshold can be determined to be the same or similar images, and they need to be deduplicated and cleaned.

[0166] One-to-many mapping output: For each semantic repetition group, one original image can be retained (usually the one with the highest quality score, or various retention rules can be referenced from the previous examples, etc., which are not limited here), and the remaining images can be marked as semantic repetitions. Then, a "one-to-many" mapping relationship can be output, the mathematical expression of which can be:

[0167] After removing duplicates from the feature-similar images (images with the same features) identified in the above examples, the cleaned dataset after deduplication can be obtained. .

[0168] Similarly, one can choose to target... Implement a classification and cleaning strategy: The data to be cleaned is classified and processed to obtain... The classification labels and classification confidence scores of each data point to be cleaned are provided. Data points with higher classification confidence scores (high-confidence samples) can be selected for training the surrogate classifier. .

[0169] Furthermore, Each piece of data to be cleaned is input into the proxy classifier. This will give you the output of the proxy classifier. The predicted labels corresponding to each piece of data to be cleaned.

[0170] Therefore, the initial classification label can be used as the original label, and loss calculation can be performed with the predicted label corresponding to the same data. Then, the presence of label noise in the data can be determined by combining the preset loss threshold, which determines whether label correction should be performed subsequently.

[0171] Step S430: If the feature analysis results indicate that the data quality of the dataset to be cleaned meets the data quality conditions and the data duplication rate of the dataset to be cleaned meets the data duplication conditions, then the classification cleaning strategy is determined as the target cleaning strategy.

[0172] When the data quality of the dataset to be cleaned meets the data quality conditions (meaning that quality cleaning is not necessary), and when the data duplication rate of the dataset to be cleaned meets the data duplication conditions (meaning that deduplication cleaning is not necessary).

[0173] Therefore, the classification and cleaning strategy can be determined as the target cleaning strategy.

[0174] Based on the above embodiments, it should also be noted that in some possible implementations, the quality cleaning strategy, deduplication cleaning strategy, and classification cleaning strategy may be determined as the target cleaning strategy, which will not be elaborated here.

[0175] Referring to the foregoing embodiments, if the original label and predicted label of the same data do not meet the preset label consistency conditions during the implementation of the classification and cleaning strategy (e.g., the label similarity between the original label and the predicted label is less than the corresponding preset anomaly threshold), then the data is data that needs to be labeled.

[0176] Optionally, the methods for classifying the data to be cleaned in this application may include one or more, and the specific method can be set as needed according to the actual application scenario, without limitation here. For example, scene classification and / or target object classification can be performed based on the image content of the image, without limitation here.

[0177] For example, this application can employ a dual classifier architecture to perform multi-dimensional classification of images, thereby simultaneously recognizing the scene and target object in the image content.

[0178] For example, the steps in this embodiment may include at least: The data to be cleaned is input into a pre-trained scene classifier and a pre-trained target classifier, respectively, to obtain the scene prediction label output by the scene classifier and the target prediction label output by the target classifier. The classification label is determined based on the scene prediction label and the target prediction label of the same data to be cleaned, and the classification confidence is determined based on the scene classification confidence corresponding to the scene prediction label and the target detection confidence corresponding to the target prediction label.

[0179] Specifically, feature extraction: Each image can be extracted using a pre-trained neural network model (such as CLIP or CNN models). eigenvectors It can be used in subsequent scene classification and object detection processes.

[0180] Scene Classification: A pre-trained scene classifier (such as Places365, CLIP scene understanding branch, etc.) is used to classify the scene of the image, and the corresponding scene category label (scene prediction label) and confidence score are output:

[0181] in, For scene categories, Confidence level for scene classification.

[0182] Object detection classification: A pre-trained object detection classifier (such as YOLO series, Faster R-CNN, etc.) is used to detect objects in the image, and the output is a list of detected object categories (object prediction labels) and the confidence score of each object:

[0183] in, For image The number of target objects detected in the process. For the first The categories of targets, The corresponding target detection confidence level, These are the coordinates of the detection box.

[0184] Optionally, in application scenarios that require comprehensive consideration of scene classification and target object classification, multi-dimensional tag aggregation can also be performed: The multidimensional classification labels for each image are obtained as follows: .

[0185] If only a certain type of application scenario needs to be considered (such as scenario classification or target object classification), then the corresponding type of method can be executed and analyzed.

[0186] Furthermore, confidence levels are assessed and low-confidence images are marked for each image: In application scenarios that comprehensively consider scene classification and target object classification, it is necessary to combine the scene classification confidence and the target detection confidence to define the overall classification confidence, which can be mathematically expressed as:

[0187] in, Alternatively, other values ​​can be set as weighting coefficients for scene classification confidence and object detection confidence. These values ​​can be set as needed based on the actual application scenario, and there are no restrictions here.

[0188] Furthermore, for cases where the overall classification confidence is lower than the classification confidence threshold... Samples with low confidence levels can be labeled with a low confidence level, and these labels can be subsequently reviewed by relevant staff.

[0189] Alternatively, multiple classification confidence thresholds can be set as needed, and samples can be stratified according to confidence levels based on these thresholds, such as classifying samples into low-confidence samples, medium-confidence samples, and high-confidence samples.

[0190] Specifically, the methods for validating the classification / clustering results to ensure classification accuracy may include at least the following: Cluster center visualization: Features such as t-SNE or UMAP can be used to reduce the dimensionality of the feature vectors and visualize them in order to check the rationality of the clustering results.

[0191] Stratified sampling based on confidence level (sampling can be performed automatically by the system and / or manually by stratifying by confidence level): For low confidence samples ( The sampling rate is ) The typical value is 50%, but other values ​​can be set as needed. and They can be the same or different; no restrictions are imposed here.

[0192] For medium confidence samples ( The sampling rate is ) The typical value is 30%, but other values ​​can be set as needed.

[0193] For high confidence samples ( The sampling rate is ) The typical value is 20%, but other values ​​can be set as needed.

[0194] The mathematical expression for the overall sampling ratio can be:

[0195] in, These represent the sample sizes for low confidence, medium confidence, and high confidence levels, respectively.

[0196] Classification accuracy assessment: Calculate the classification accuracy based on the sampling results: .in, This refers to the number of correctly classified samples in the overall random sampling. This refers to the number of samples in the comprehensive sampling that have undergone accuracy checks. All or part of the quantity.

[0197] If the accuracy does not meet the standard (e.g., the classification accuracy is less than the classification accuracy threshold), (Typical value is 0.90, or other values ​​can be set as needed). You can choose to adjust the classification model by prompt words (for CLIP zero-sample classification) or fine-tune the training to optimize the classification model, and then re-execute the multi-dimensional classification in the above embodiment.

[0198] If the accuracy meets the requirements, label noise detection and cleaning can be performed on the classified / clustered labeled dataset to ensure the label accuracy of the training data (avoiding problems such as mislabeling and omissions). This is especially suitable for datasets with manual labeling errors or automatic labeling noise.

[0199] Specifically, train the proxy classifier: A clean subset can be extracted from the classified dataset in the aforementioned embodiments. (For example, based on high-confidence samples determined in the aforementioned embodiments, or high-confidence samples and medium-confidence samples, etc.), train a lightweight surrogate classifier. .

[0200] On the one hand, the small-loss criterion can be used to analyze whether the labels are abnormal, such as: For each image Calculate the training loss of the proxy classifier Its mathematical expression can be:

[0201] in, For the sample The original labels (the classification labels obtained by classification in the previous example).

[0202] Therefore, the mean of the loss distribution can be calculated. and standard deviation Define the adaptive loss threshold: .

[0203] in, The adjustment coefficient can be preset, or other values ​​can be set as needed. Therefore, the criteria for determining the presence of noise in the label can be set as follows: To identify images with noise labels (i.e., labeled noise data).

[0204] Alternatively, confidence learning can be used to analyze whether labels are abnormal, such as: For each image The predicted probability of the surrogate classifier can be calculated: Its corresponding original tag Label consistency score between: This refers to the probability predicted by the classification model for the original label.

[0205] Among them, the label consistency score is compared with the preset label consistency threshold. To make a comparison, if Then, samples with low consistency can be identified as mislabeled samples (the labels contain noise). Alternatively, it can be set to other values.

[0206] It should be noted that in the aforementioned examples, the label analysis methods based on the small loss criterion and confidence learning can be selected to be executed, or both can be executed; there is no limitation here. For example, the method based on the small loss criterion can be executed first to analyze label noise, and then the method based on confidence learning can be executed to analyze label noise. The union or intersection data obtained by the two methods can be selected as the samples with label noise; there is no limitation here.

[0207] Tag correction: For samples with noisy labels in at least one of the aforementioned examples Automatic label correction based on the prediction results of the surrogate classifier:

[0208] Alternatively, it can be marked as "awaiting manual review," with the final label to be confirmed manually; no specific restrictions are imposed here.

[0209] This allows for the cleaning of images with noisy labels.

[0210] Furthermore, a noise rate assessment can then be performed, including: Calculate the label noise rate of the cleaned dataset:

[0211] in, This refers to images with noisy labels. This refers to images with labels. If the label noise rate exceeds a preset label noise threshold... The typical value is 0.10, but it can be set to other values. This will trigger a message prompt suggesting a review of the annotation process used when building the dataset to be cleaned.

[0212] Based on the above embodiments, this embodiment describes the method in step S130.

[0213] Specifically, the method for arranging the target cleaning strategy to obtain the cleaning task in this embodiment includes at least the following steps S510 to S520: Step S510: Obtain the strategy complexity of each target cleaning strategy.

[0214] Step S520: Arrange the cleaning strategies for each target in ascending order of strategy complexity to obtain the cleaning task.

[0215] In conjunction with the foregoing embodiments, in a scenario where multiple target cleaning strategies are selected, the strategy complexity of each target cleaning strategy can be obtained (which may be preset or obtained after at least one experiment or actual operation, and is not limited here).

[0216] Among them, the target cleaning strategies can be arranged according to the ascending order and the strategy complexity (the same logic can be applied to the target cleaning sub-strategies).

[0217] Optionally, the granularity of complexity settings can be selected based on sub-policies. That is, the complexity of a target cleaning policy can also be determined based on its constituent target cleaning sub-policies (e.g., by summation, etc., which will not be elaborated here).

[0218] For example, in some application scenarios, the complexity of quality cleaning can be less than that of deduplication cleaning, and the complexity of deduplication cleaning can be less than that of label cleaning. Therefore, the execution order of each target cleaning strategy in the cleaning task obtained by orchestration processing can be: 1. Quality cleaning; 2. Deduplication cleaning; 3. Label cleaning.

[0219] For example, in other application scenarios, the complexity of deduplication cleaning can be less than that of quality cleaning, and the complexity of quality cleaning can be less than that of label cleaning. Therefore, the execution order of each target cleaning strategy in the cleaning task obtained by orchestration processing can be: 1. Deduplication cleaning; 2. Quality cleaning; 3. Label cleaning.

[0220] Based on the above embodiments, this embodiment should also be noted that, when arranging the target cleaning strategies in this application, it also includes allocating corresponding computing resources for each target cleaning strategy.

[0221] For example, computational resources can be allocated to each target cleaning strategy by comparing the percentage of defects corresponding to each target cleaning strategy (i.e., the computational resources occupied by each cleaning strategy or sub-strategy are positively correlated with the percentage of defects). Alternatively, the execution priority (execution order) of each target cleaning strategy can be determined by the percentage of defects.

[0222] For example, in (The blurred image needs to be cleaned), and If the proportion of blurred images is greater than the proportion of noisy images, then more computing resources can be allocated to the strategy of blurry image cleaning.

[0223] Based on the above embodiments, this embodiment describes the method in step S140.

[0224] The cleaning task includes label cleaning task, or includes at least one of label cleaning task, low-quality cleaning task, and deduplication cleaning task.

[0225] Specifically, the method for performing a cleaning task on the dataset to be cleaned and obtaining the data cleaning result in this embodiment includes at least the following steps S610 to S640: Step S610: Perform a label cleaning task on the dataset to be cleaned, or perform at least one of a low-quality cleaning task and a deduplication cleaning task, as well as a label cleaning task, on the dataset to be cleaned to obtain the data cleaning result.

[0226] Step S620, wherein the low-quality cleaning task includes: determining low-quality data in the dataset to be cleaned based on the image attributes of the dataset to be cleaned; and filtering the low-quality data.

[0227] Step S630, and / or, the deduplication and cleaning task includes: determining duplicate data in the dataset to be cleaned based on the image similarity between each data in the dataset to be cleaned; and performing deduplication processing on the duplicate data.

[0228] Step S640, and / or, the label cleaning task includes: classifying the data to be cleaned in the dataset to be cleaned to obtain the classification label and classification confidence of the data to be cleaned; selecting target data from the data to be cleaned based on the classification confidence to train a surrogate classifier to obtain a surrogate classifier; inputting the data to be cleaned into the surrogate classifier to obtain the predicted label of the data to be cleaned; determining noise labels based on the consistency between the classification label and the predicted label of the same data to be cleaned; and correcting the noise labels.

[0229] In conjunction with the foregoing embodiments, the cleaning task includes at least a label cleaning task that executes a classification cleaning strategy. In addition, based on the feature analysis results of the dataset to be cleaned, it is also possible to select whether to execute a low-quality cleaning task with a quality cleaning strategy and / or a deduplication cleaning task with a deduplication cleaning strategy.

[0230] For example, reference can be made to, such as Figure 2 As shown, Figure 2 This is a schematic diagram of an exemplary data cleaning system in the image dataset data cleaning method of this application. It includes the three cleaning modules of the example cleaning strategy described above, as well as modules for evaluating the cleaning results and for providing feedback enhancement to the cleaned image data.

[0231] The specific implementation process of various cleaning tasks in this embodiment can be referred to the explanation of various cleaning strategies in the foregoing embodiments, and will not be repeated here.

[0232] Based on the above embodiments, this embodiment should also explain that when the cleaning task of the aforementioned example is completed each time, the execution result (data cleaning result) of the cleaning task can be verified to verify whether the cleaning task is qualified (whether the cleaned dataset meets the preset verification conditions), so as to determine whether the cleaning parameters of each type of cleaning task need to be adjusted.

[0233] Specifically, in this embodiment, the method after performing a cleaning task on the data to be cleaned and obtaining the data cleaning result includes at least the following steps S710 to S740: Step S710: Perform cleaning effect verification processing on the low-quality cleaning task to obtain the first verification result.

[0234] Step S720: If the first verification result does not meet the preset first verification condition, then adjust the low-quality cleaning parameters of the low-quality cleaning task according to the first verification result.

[0235] Step S730, and / or, perform cleaning effect verification processing on the deduplication cleaning task to obtain a second verification result.

[0236] Step S740: If the second verification result does not meet the preset second verification condition, then adjust the deduplication cleaning parameters of the deduplication cleaning task according to the second verification result.

[0237] For example, different cleaning tasks can have different verification conditions.

[0238] For example, the first verification result corresponding to a low-quality cleaning task can be verified using the first verification condition. The second verification result corresponding to a deduplication cleaning task can be verified using the second verification condition.

[0239] If the verification fails, the cleaned dataset will still be temporarily unusable. The dataset can be adjusted or cleaned again, and there are no restrictions here.

[0240] The data cleaning result can refer to the result after each cleaning strategy is executed, and / or it can refer to the result after all cleaning tasks are executed; there is no limitation here.

[0241] Optionally, in this application, a verification can be performed after each target cleaning strategy (or cleaning sub-strategy) has been executed, and / or a verification can be performed after all target cleaning strategies (the complete cleaning task) have been executed; no limitation is made here.

[0242] For example, in some implementations of a quality cleaning strategy, the results of filtering low-quality images (which may include retained images and / or filtered images) may be verified to ensure the rationality of the cleaning parameters in the quality cleaning strategy.

[0243] Specifically, this can be achieved, including but not limited to, through manual or automated random inspections.

[0244] For example: from the filtered image set Randomly selected from The images are reviewed to verify whether they are indeed low-quality images. The false deletion rate can be calculated.

[0245] in, This represents the number of valid images mistakenly identified as low-quality images. Under the condition that the low-quality image cleaning strategy is executed correctly, the false deletion rate must meet the following requirements:

[0246] in, This is the preset upper limit threshold for the false deletion rate. The typical value is 0.05, or it can be set to other values.

[0247] For example, it could also include injection-based defect recall testing: injection The pre-defined known defective images (which may include blurred images, overexposed images, high-noise images, etc.) are added to the dataset to be cleaned before cleaning and filtering (e.g., ...). ,or wait).

[0248] After the cleaning and filtering process of the low-quality cleaning strategy described in the previous example, we analyze whether all the injected known defect images can be detected, and calculate the defect recall rate:

[0249] in, The number of images with injection defects detected. Under the condition that the low-quality cleaning strategy is correctly executed, the recall rate must meet the following requirements:

[0250] in, This is the preset recall threshold, typically set to 0.95 or other values.

[0251] Based on the first verification results of the low-quality cleaning strategy (false deletion rate and / or defect recall rate) and the corresponding first verification conditions (preset upper threshold for false deletion rate and / or lower threshold for recall rate), determine whether to perform parameter rollback adjustment: like or If not satisfied, then revert to adjusting the filtration intensity parameters (e.g.) , (etc.) are parameters from the previous version, or default values, or adjustments made according to other preset rollback rules, etc.

[0252] Then, you can choose to re-execute the low-quality cleaning strategy until the verification passes, or generate an error message if it still fails after reaching a preset number of executions. Specific settings can be configured according to the actual application scenario; no limitations are specified here.

[0253] For example, in some implementations of a deduplication cleaning strategy, the deduplication image filtering results (which may include retained images and / or filtered images) may be verified to ensure the rationality of the deduplication threshold in the deduplication cleaning strategy.

[0254] The verification method may include one or more dimensions, such as: performing quantity statistics and repetition rate calculation on images before and after deduplication, including: The change in the number of images before and after deduplication is statistically analyzed, and the duplication rate is calculated. Its mathematical expression can be:

[0255] Manual or automatic spot checks are performed based on the one-to-many mapping results: Randomly sample from "one-to-many" mapping The group was reviewed to verify whether it was indeed a duplicate problem (such as pixel duplication, semantic duplication, etc.).

[0256] The false positive rate can be calculated from this:

[0257] Under the condition that the deduplication and cleaning strategy is executed correctly, the false positive rate must meet the following requirements:

[0258] in, This is the preset upper limit threshold for the false positive rate, typically set to 0.10, or other values ​​can be set as needed.

[0259] Similarly, injection-based duplicate recall testing can also be included: injection For a dataset containing known duplicate images before deduplication, check whether all duplicate images can be detected after the deduplication process.

[0260] The mathematical expression for repeat recall can be:

[0261] Under the condition that the deduplication and cleaning strategy is executed correctly, the duplicate recall rate must meet the following requirements:

[0262] in, This is the preset recall lower limit threshold, typically set to 0.95, or other values ​​can be set as needed.

[0263] It can also perform sampling analysis based on the similarity matrix, including at least: The cleaned data corresponding to the deduplication and cleaning task is sampled to obtain deduplication and cleaning sampled data; a similarity matrix is ​​obtained between every two deduplication and cleaning sampled data; based on the similarity of a specified region in the similarity matrix, deduplication and cleaning similar data with a similarity greater than a preset feature similarity threshold are determined; the deduplication and cleaning similarity ratio is determined based on the number of deduplication and cleaning sampled data and the number of deduplication and cleaning similar data; the second verification result is determined based on the comparison between the deduplication and cleaning similarity ratio and the preset deduplication and cleaning similarity ratio threshold.

[0264] Specifically, this could involve randomly sampling the cleaned data obtained after the deduplication and cleaning task to obtain... Images (typically 100-1000, or other values ​​can be set as needed; the sampling number may be the same as or different from the sampling number in the aforementioned embodiments), repeated. Rounds (typically 5-10, or other values ​​can be set as needed) are used to calculate the similarity matrix between every two deduplicated and cleaned sampled data:

[0265] in, Check if the similarity value of a specified region (such as the region outside the diagonal) in the similarity matrix exceeds the feature similarity threshold. The proportion of duplicates (the percentage of similar items after deduplication and cleaning) can be mathematically expressed as:

[0266] like If the value is too high (e.g., greater than 0.20, or other values ​​can be set as needed), it indicates that the feature similarity threshold is set too low and needs to be increased. Conversely, the same logic applies to making the opposite adjustment, which will not be elaborated here.

[0267] It should be noted that one or more of the validation methods for the multiple dimensions in the above examples can be executed simultaneously; this is not a limitation. The same applies to other validation methods, which will not be elaborated upon here.

[0268] Parameter rollback adjustment: If the second verification result ( , The corresponding second verification condition is not met or For issues such as excessively high thresholds, you can choose to roll back and adjust the threshold in the deduplication and cleaning strategy. Alternatively, you can choose to re-execute the deduplication and cleaning process until the verification passes; or, if it still fails after reaching a preset number of executions, an error message will be generated. Specific settings can be configured according to actual application scenarios; no limitations are specified here.

[0269] It is understandable that the verification methods in the above examples can also be applied to scenarios involving hash deduplication and cosine similarity deduplication, which will not be elaborated here.

[0270] For example, in some implementation methods, the accuracy of classification can be verified by referring to the foregoing embodiments for classification cleaning strategies, which will not be elaborated here.

[0271] Based on the above embodiments, this embodiment describes the method in step S710.

[0272] Specifically, in this embodiment, the method for verifying the cleaning effect of a low-quality cleaning task and obtaining a first verification result includes at least the following steps S810 to S820: Step S810: Negative data injection processing is performed on the pre-cleaning data corresponding to the low-quality cleaning task to obtain the low-quality cleaning verification dataset.

[0273] Step S820: Perform a low-quality cleaning task and recall calculation on the low-quality cleaning and verification dataset to obtain the first verification result.

[0274] Referring to the foregoing embodiments, if the data cleaning results of the low-quality cleaning strategy are to be verified, the negative data injection process can refer to the method of injecting defective images in the foregoing embodiments (it is possible to inject defective images corresponding to low-quality defects) to obtain the low-quality cleaning verification dataset corresponding to the low-quality cleaning strategy.

[0275] Then, similarly, the low-quality cleaning strategy and recall rate verification can be performed on the low-quality cleaning verification dataset according to the method of the aforementioned embodiments, and the data recall rate (which can be used as the first verification result) is obtained.

[0276] The data recall rate is compared with the corresponding preset recall rate threshold (first verification condition) to obtain the quality cleaning result.

[0277] Similarly, when performing cleaning verification on a deduplication cleaning task to obtain a second verification result, the steps may include: Negative data injection is performed on the pre-cleaning data corresponding to the deduplication and cleaning task to obtain the deduplication and cleaning verification dataset; the deduplication and cleaning task and recall rate are calculated on the deduplication and cleaning verification dataset to obtain the second verification result.

[0278] Based on the aforementioned steps, if the data cleaning results of the deduplication cleaning strategy are to be verified, the negative data injection process can refer to the method of injecting duplicate image pairs in the aforementioned embodiments to obtain the deduplication cleaning verification dataset of the deduplication cleaning strategy.

[0279] Then, the deduplication cleaning strategy and recall rate verification can be performed on the deduplication cleaning verification dataset according to the method of the aforementioned embodiments, and the corresponding data recall rate (which can be used as the second verification result) can be obtained.

[0280] The data recall rate is compared with the corresponding preset recall rate threshold (second verification condition) to obtain the deduplication and cleaning results.

[0281] Based on the above embodiments, this embodiment describes the method of step S150.

[0282] Specifically, the method for determining whether to perform data optimization processing on the cleaned dataset based on the quality analysis report in this embodiment includes at least the following steps S910 to S930: Step S910: Determine the quality score of each cleaned data in the cleaned dataset based on the quality analysis report.

[0283] Step S920: Determine the data to be optimized from each cleaned data based on the quality score.

[0284] Step S930: Perform image enhancement and optimization processing on the data to be optimized.

[0285] In conjunction with the foregoing embodiments, this application can reflect the quality of the overall cleaning result by evaluating and verifying the data (cleaned data) after the entire cleaning task has been completed.

[0286] In the process of quality assessment, the cleaned data can be scored using methods including but not limited to no reference, full reference, partial reference, or subjective evaluation to obtain a quality score for the cleaned data and record it in the quality analysis report.

[0287] Then, based on the quality score, it can be determined whether there is data to be optimized in each cleaned dataset (such as data with a quality score lower than a preset score threshold). If so, image enhancement and optimization processing is performed on it.

[0288] Based on the above embodiments, this embodiment describes the method of step S150.

[0289] Specifically, the method for determining whether to perform data optimization processing on the cleaned dataset based on the quality analysis report in this embodiment includes at least the following steps S1010 to S1020: Step S1010: Determine the number of samples corresponding to each data type in the cleaned dataset based on the quality analysis report.

[0290] Step S1020: For data types with fewer than a preset sample number threshold, perform data supplementation and optimization processing.

[0291] Since the cleaned dataset has undergone classification processing, the quality analysis report can include the number of samples corresponding to each data type in the cleaned dataset.

[0292] Understandably, if the number of samples of a certain type is small (e.g., less than a preset sample number threshold), then it is necessary to supplement the sample data of that type in order to optimize the sufficiency of the dataset.

[0293] For example, in some possible implementations, each image can be scored based on a No-Reference Image Quality Assessment (No-Reference IQA) method, including: (1) Quality scoring model Multimodal large language models (MLLMs, such as those based on DeQA-Score) can be used for each image. Conduct quality scoring, such as:

[0294] in, The quality score is a positive correlation; a higher score indicates better image quality.

[0295] Optionally, MLLM can be replaced with other IQA methods (such as BRISQUE, NIQE, PIQE, etc.) as needed, and there is no limitation here.

[0296] (2) Multidimensional quality score In addition to the overall quality score, multiple quality indicators such as sharpness, brightness, and noise can be evaluated separately.

[0297] (3) Weighted composite score Based on preset rules or downstream task requirements in actual application scenarios, the quality scores of each dimension are weighted to obtain a comprehensive score for image quality:

[0298] in, These are the weighted weights for each dimension indicator. The weights for different indicators can be the same or different; there is no restriction here. .

[0299] For example, the quality analysis report may include: a comparison of the number of data in the dataset before and after cleaning, the number of samples in each category, the image quality distribution, and records of cleaning parameters, etc., without limitation here.

[0300] For example, the quality scoring process may include: summarizing the cleaning results, visualizing the classification statistics, weighting the multidimensional quality scores, and generating a dataset quality analysis report.

[0301] (1) Statistics of cleaning results: Count the number of images filtered (cleaned) in the execution stage of each target cleaning strategy: The overall cleaning rate is calculated, and its mathematical expression can be:

[0302] (2) Classification / Scene Distribution Statistics: By counting the number of samples in each category / cluster, visual data such as histograms or pie charts can be generated.

[0303] (3) Image quality distribution statistics: Image quality scores for each image data after statistical cleaning The distribution (can be characterized by forms such as histograms and box plots).

[0304] (4) Comprehensive Report: Summarize the above statistical information to generate a dataset quality analysis report.

[0305] Furthermore, the quality analysis report can be used to determine the distribution balance and quantity sufficiency of the cleaned dataset, and targeted enhancements can be performed. For example: (1) Judgment of distribution balance: Calculate the ratio of the standard deviation to the mean of the sample size for each category / cluster (coefficient of variation): ,like (A preset distribution balance threshold, typically 0.5 or other values ​​can be set as needed), indicates that the cleaned dataset has an imbalance in the distribution of sample types.

[0306] (2) Quantity sufficiency judgment: Determine if the number of samples in each category / cluster is sufficient to meet the quantity requirements of downstream tasks: ,in A minimum sample size threshold is preset for each category.

[0307] (3) High-quality image supplementation: If the number of samples for a certain category is insufficient, high-quality images can be supplemented by transforming the data in the cleaned dataset (such as flipping, rotating, cropping, mixing, etc.) or by obtaining high-quality images from external datasets.

[0308] (4) Image enhancement optimization: For low-quality images, targeted enhancement methods can be adopted based on the anomalous indicators (anomalous types) of the low-quality images, such as including but not limited to: denoising (e.g., Guided Filtering, FFDNet, etc.), motion blur removal (e.g., IDR (Iterative Deblurring with Residual), etc.), brightness / contrast adjustment (e.g., histogram equalization, CLAHE, etc.).

[0309] Enhanced image quality score Must meet:

[0310] in, It is the image quality score before enhancement (e.g.) ). Increase the threshold for the preset quality score.

[0311] Optionally, the enhancement method can be replaced with other enhancement methods (such as DnCNN, WDNet, etc.) as needed, and there is no limitation here.

[0312] Based on the above embodiments, this embodiment should also explain that after obtaining the data cleaning results, the effectiveness of the cleaning can be verified by using the downstream task (preset model training task) of the dataset to be cleaned.

[0313] Specifically, this embodiment may include at least steps S1101 to S1102: Step S1101: Verify the cleaning effectiveness of the cleaned dataset in the data cleaning results according to the preset model training task to obtain the cleaning effectiveness of the cleaned dataset.

[0314] Step S1102: If the cleaning effectiveness does not meet the preset cleaning effectiveness conditions, adjust the cleaning parameters of the target cleaning strategy.

[0315] For example, using the mAP (Mean Average Precision) metric of downstream target detection tasks as a criterion for cleaning effectiveness may include: (1) Division of training and test sets: Based on the quality analysis report, the training set can be divided according to the proportion of scenarios / categories. With test set The typical ratio is 8:2 or 7:3, or other ratios can be set as needed.

[0316] (2) Baseline training before cleaning: Train an object detection model (such as YOLOv8) on the original unwashed dataset and evaluate mAP on the test set:

[0317] Optionally, during the downstream task validation process, YOLOv8 can also replace the object detection model with other models (such as Faster R-CNN, DETR, etc.) or replace it with an image classification task for validation as needed; no limitation is made here.

[0318] (3) Model training after cleaning: Train the object detection model using the cleaned dataset to be cleaned, and evaluate mAP on the same test set as in the previous example:

[0319] (4) Determination of cleaning effectiveness: The mathematical expression for calculating the increase in mAP as a measure of cleaning effectiveness can be:

[0320] Effective cleaning conditions can be: .

[0321] in, The threshold for increasing mAP is set to a preset value, typically ranging from 0.01 to 0.05, or other values ​​can be set as needed.

[0322] (5) Feedback rollback: like If the validity determination conditions are not met, the methods for not meeting the preset conditions in the aforementioned embodiments can be referred to: select to roll back and adjust the filter intensity parameters, deduplication threshold, or enhancement strategy, etc.; or select to re-execute the cleaning steps and verification steps of the aforementioned embodiments until the verification is effective or the preset number of times threshold is reached.

[0323] Based on the above embodiments, this embodiment further explains that after training the relevant neural network model using the cleaned dataset, the method in this application can further identify difficult examples in the cleaned dataset and analyze their quality characteristics, forming a deep closed loop. Here, difficult examples refer to samples that are difficult for neural network models to classify correctly in machine learning.

[0324] Specifically, this embodiment may include at least steps S1201 to S1203: Step S1201: Identify the hard examples in the cleaned dataset when performing the preset model training task, based on the preset confidence threshold and loss threshold.

[0325] Step S1202: Adjust the cleaning parameters of the target cleaning strategy according to the image features of the difficult sample.

[0326] Step S1203: Mark and retain the key difficult sample containing the key target.

[0327] For example, when performing a pre-defined model training task, the training confidence and training loss corresponding to each cleaned data point in the cleaned dataset can be obtained. Therefore, cleaned data points with training confidence less than a confidence threshold and / or training loss greater than a loss threshold can be identified as hard examples.

[0328] Then, the cleaning parameters (corresponding thresholds) of various target cleaning strategies in the aforementioned examples can be adjusted based on the image features of the difficult sample (such as sharpness, image noise, size distribution, etc.).

[0329] Specifically, difficult case identification: For the training set Each sample The calculation model's prediction confidence level and training loss The criteria for classifying a sample as a difficult example can be set as follows: or .

[0330] in, The confidence threshold. (Or you can set it to other values ​​as needed). This is the loss threshold (which can be the 90th percentile of the training loss distribution or other values).

[0331] Analysis of Quality Characteristics of Difficult Examples: The set of difficult examples obtained through difficult example identification The distribution of its quality features is statistically analyzed. The quality features can refer to several features used to evaluate image quality as described in the previous embodiments (such as sharpness, image size, image noise, etc., and for target detection models, the proportion of target occlusion, etc.).

[0332] For example, analyzing the distribution of quality characteristics of difficult examples yields: sharpness distribution (the proportion of blurry samples in difficult examples). ), noise level distribution (proportion of image noise samples in difficult examples), ), Target size distribution (proportion of small targets in difficult examples) ), distribution of occlusion ratio (percentage of difficult cases with occlusion) )wait.

[0333] Inverse parameter adjustment: Based on the quality characteristics analysis of difficult examples, it can be determined whether key training samples were lost due to excessive cleaning.

[0334] For example, taking the resolution distribution ratio as an example, if (e.g., 0.30 or other values ​​can be set as needed), indicating that slightly blurred samples have some value for model training, and the filtering threshold for blurred images in low-quality cleaning strategies can be relaxed. To retain more slightly blurred but key-object samples, its mathematical expression can be:

[0335] in, To adjust the step size, Alternatively, other values ​​can be set as needed. It is understandable that the parameters in the cleaning strategy corresponding to other feature distributions can also be adjusted similarly (e.g., the threshold in the deduplication strategy). (etc.), which will not be elaborated here.

[0336] Difficult example retention strategy: Difficult sample samples in the difficult sample set whose quality score is lower than the corresponding preset normal threshold but contain key objectives can be marked as "key difficult samples" and retained to avoid being accidentally deleted in the cleaning process.

[0337] Here, "key targets" can refer to the target objects and / or target events that target detection needs to focus on, without any limitation. That is, target detection or manual identification can be performed on difficult sample samples, and difficult sample samples with target objects and / or target events can be identified as key difficult sample samples for labeling and retention.

[0338] In summary, through the data processing methods of one or more embodiments described in the foregoing embodiments of this application, a high-quality target dataset (which may include a training set and a test set) can be output. Specifically: (1) Target dataset The cleaned dataset that meets the validity verification requirements of downstream tasks Can be output as: .

[0339] (2) Training set and test set The training set and test set can be determined from the target dataset and output according to the division ratio in the aforementioned embodiments: .

[0340] (3) Metadata Records Metadata of the data cleaning process in the aforementioned embodiments can be recorded, such as including but not limited to: cleaning parameters of each stage, cleaning quantity (filtering / deduplication quantity), quality distribution, downstream task mAP, etc., to facilitate subsequent reproduction and auditing.

[0341] In summary, the data cleaning method of this application can provide an end-to-end, multi-stage, progressive cleaning pipeline for deep learning training datasets. Furthermore, the pipeline stages can be dynamically orchestrated based on the characteristics of the dataset to be cleaned, avoiding unnecessary computation and cleaning.

[0342] In the cleaning process, in addition to considering image quality issues (such as blurring, noise, and duplication), the impact of label noise on model training is also taken into account. By introducing a label noise detection and cleaning module, a dual mechanism of small loss criterion and confidence learning is adopted to identify and correct erroneous label samples, filling the gap in label quality control in data cleaning scenarios.

[0343] The distribution balance of the data in the dataset to be cleaned is incorporated into the cleaning closed loop, and data augmentation and optimization are automatically performed.

[0344] Furthermore, after executing the target cleaning strategy, a closed-loop verification mechanism can be set up for injection-type defect / repeat recall rate testing and stratified sampling to adaptively adjust the parameters of the cleaning strategy.

[0345] After the cleaning task is completed, the actual performance improvement of downstream tasks (such as the mAP index of the YOLO model) can be used as the final criterion for the effectiveness of cleaning, forming an adaptive closed loop of "cleaning-verification-feedback-re-cleaning".

[0346] Furthermore, a difficult example mining and feedback mechanism is introduced. By identifying difficult examples in model training and adjusting the cleaning parameters in reverse, the key training information is avoided from being lost due to excessive cleaning.

[0347] It should be further noted that the entity executing the image dataset data cleaning method can be an image dataset data cleaning device. For example, the image dataset data cleaning method can be executed by a terminal device, server, or other processing device. The terminal device can be a user equipment (UE), computer, mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, the image dataset data cleaning method can be implemented by a processor calling computer-readable instructions stored in memory.

[0348] Figure 3This is a block diagram illustrating a data cleaning apparatus for an image dataset, as shown in an exemplary embodiment of this application. Figure 3 As shown, the exemplary image dataset data cleaning apparatus 300 includes: a data analysis module 310, a strategy determination module 320, a strategy orchestration module 330, and a cleaning execution module 340. Specifically: The data analysis module 310 is used to perform sampling feature analysis on the acquired dataset to be cleaned and obtain feature analysis results.

[0349] The strategy determination module 320 is used to determine the target cleaning strategy from a preset cleaning strategy set based on the feature analysis results.

[0350] The strategy orchestration module 330 is used to orchestrate the target cleaning strategy to obtain the cleaning task.

[0351] The cleaning execution module 340 is used to perform cleaning tasks on the dataset to be cleaned and obtain the data cleaning results.

[0352] In this exemplary image dataset data cleaning device, the acquired dataset to be cleaned undergoes sampling feature analysis processing to obtain feature analysis results. These results typically characterize any abnormal defects present in the dataset. Based on these results, a suitable target cleaning strategy can be determined from a preset set of cleaning strategies. The selected target cleaning strategies are then arranged to obtain the cleaning task. By executing the cleaning task on the data to be cleaned, the data cleaning result, i.e., the cleaned dataset, can be obtained. This achieves the selection of a suitable cleaning strategy based on the actual defects of the data to be cleaned, significantly improving data cleaning efficiency. It should be noted that the device and method provided in the above embodiments belong to the same concept. The specific methods by which each module and unit performs operations have been described in detail in the method embodiments and will not be repeated here. In practical applications, the device provided in the above embodiments can allocate the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above. This is not a limitation.

[0353] The functions of each module can be found in the implementation examples of the data cleaning method for image datasets, and will not be repeated here.

[0354] Please see Figure 4 , Figure 4This is a schematic diagram of the structure of an embodiment of the electronic device of this application. The electronic device 100 includes a memory 101 and a processor 102. The processor 102 is used to execute program instructions stored in the memory 101 to implement the steps in any of the above-described embodiments of the data cleaning method for image datasets. In a specific implementation scenario, the electronic device 100 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 100 may also include mobile devices such as laptops and tablets, which are not limited here.

[0355] Specifically, processor 102 controls itself and memory 101 to implement the steps in any of the above-described image dataset data cleaning method embodiments. Processor 102 can also be referred to as a CPU (Central Processing Unit). Processor 102 may be an integrated circuit chip with signal processing capabilities. Processor 102 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 102 can be implemented using integrated circuit chips.

[0356] In this exemplary electronic device, feature analysis is performed on the acquired dataset to be cleaned to obtain feature analysis results. These results typically characterize any abnormal defects present in the dataset. Based on these results, a suitable target cleaning strategy can be determined from a pre-defined set of cleaning strategies. The selected target cleaning strategies are then orchestrated to obtain the cleaning task. By executing the cleaning task on the data to be cleaned, the cleaned dataset is obtained. This process significantly improves data cleaning efficiency by enabling the selection and orchestration of suitable cleaning strategies based on the actual defects in the data.

[0357] Please see Figure 5 , Figure 5 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 110 stores program instructions 111 that can be executed by a processor. The program instructions 111 are used to implement the steps in any of the above-described embodiments of the data cleaning method for image datasets.

[0358] In this exemplary storage medium, by running program instructions within the storage medium, the acquired dataset to be cleaned undergoes sampling feature analysis processing to obtain feature analysis results. These results typically characterize any abnormal defects present in the dataset. Based on these results, a suitable target cleaning strategy can be determined from a pre-defined set of cleaning strategies. The selected target cleaning strategies are then orchestrated to obtain the cleaning task. By executing the cleaning task on the data to be cleaned, the cleaned dataset is obtained. This process enables the selection and orchestration of suitable cleaning strategies based on the actual defects of the data to be cleaned, significantly improving data cleaning efficiency.

[0359] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0360] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0361] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0362] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A data cleaning method for an image dataset, characterized in that, The method includes: The obtained dataset to be cleaned is subjected to sampling feature analysis to obtain feature analysis results, which are used to characterize the abnormal defects of the dataset to be cleaned; Based on the feature analysis results, a target cleaning strategy is determined from a preset set of cleaning strategies; The target cleaning strategy is arranged to obtain the cleaning task; The step of orchestrating the target cleaning strategies to obtain cleaning tasks includes: orchestrating according to the strategy complexity of each target cleaning strategy to obtain the cleaning tasks; The cleaning task is performed on the dataset to be cleaned to obtain the data cleaning result.

2. The method according to claim 1, characterized in that, The sampling feature analysis of the acquired dataset to be cleaned, to obtain the feature analysis results, includes: The dataset to be cleaned is sampled to obtain sampled data; The sampled data is subjected to feature analysis to obtain the data features of the sampled data; Anomaly analysis is performed on the sampled data based on the data characteristics to obtain the feature analysis results.

3. The method according to claim 2, characterized in that, The step of performing anomaly index analysis on the sampled data based on the data characteristics to obtain the feature analysis results includes: Based on the data characteristics, abnormal data in the sampled data are determined, and the abnormal data includes at least one of low-quality data and duplicate data. Based on the number of anomalies in each type of abnormal data and the number of samples in the sampled data, determine the anomaly index corresponding to each type of abnormal data; The abnormal indicators are compared with preset abnormal thresholds to obtain the feature analysis results.

4. The method according to claim 1, characterized in that, The cleaning strategy set includes at least a quality cleaning strategy, a deduplication cleaning strategy, and a classification cleaning strategy. The step of determining the target cleaning strategy from the preset cleaning strategy set based on the feature analysis results includes: If the feature analysis results indicate that the data quality of the dataset to be cleaned does not meet the preset data quality conditions, then the quality cleaning strategy and the classification cleaning strategy are determined as the target cleaning strategy. If the feature analysis results indicate that the data duplication rate of the dataset to be cleaned does not meet the preset data duplication conditions, then the deduplication cleaning strategy and the classification cleaning strategy are determined as the target cleaning strategy. If the feature analysis results indicate that the data quality of the dataset to be cleaned meets the data quality conditions, and the data duplication rate of the dataset to be cleaned meets the data duplication conditions, then the classification cleaning strategy is determined as the target cleaning strategy.

5. The method according to claim 1, characterized in that, The process of orchestrating the target cleaning strategy to obtain the cleaning task includes: Obtain the strategy complexity of the cleaning strategy for each target; The cleaning strategies for each target are arranged and processed in ascending order of strategy complexity to obtain the cleaning task.

6. The method according to claim 1, characterized in that, The cleaning task includes a label cleaning task, or at least one of a low-quality cleaning task and a deduplication cleaning task, plus the label cleaning task. Performing the cleaning task on the dataset to be cleaned to obtain the data cleaning result includes: The data cleaning result is obtained by performing the label cleaning task on the dataset to be cleaned, or by performing at least one of the low-quality cleaning task and the deduplication cleaning task, as well as the label cleaning task, on the dataset to be cleaned; wherein, The low-quality cleaning task includes: determining low-quality data in the dataset to be cleaned based on image attributes; and filtering the low-quality data. And / or, the deduplication and cleaning task includes: determining duplicate data in the dataset to be cleaned based on the image similarity between each data to be cleaned in the dataset to be cleaned; and performing deduplication processing on the duplicate data; And / or, the label cleaning task includes: classifying the data to be cleaned in the dataset to be cleaned to obtain the classification label and classification confidence of the data to be cleaned; selecting target data from the data to be cleaned based on the classification confidence to train a surrogate classifier to obtain the surrogate classifier; inputting the data to be cleaned into the surrogate classifier to obtain the predicted label of the data to be cleaned; determining noise labels based on the consistency between the classification label and the predicted label of the same data to be cleaned; and correcting the noise labels.

7. The method according to claim 6, characterized in that, The step of classifying the data to be cleaned in the dataset to be cleaned to obtain the classification labels and classification confidence scores of the data to be cleaned includes: The data to be cleaned is input into a pre-trained scene classifier and a pre-trained target classifier, respectively, to obtain the scene prediction label output by the scene classifier and the target prediction label output by the target classifier. The classification label is determined based on the scene prediction label and the target prediction label of the same data to be cleaned, and the classification confidence is determined based on the scene classification confidence corresponding to the scene prediction label and the target detection confidence corresponding to the target prediction label.

8. The method according to claim 6, characterized in that, The duplicate data includes first duplicate data and second duplicate data, and the duplicate data in the dataset to be cleaned is determined based on the image similarity between each data to be cleaned in the dataset to be cleaned. The process of removing duplicate data includes: Each piece of data to be cleaned is hashed to obtain its hash value. The hash Hamming distance between every two pieces of data to be cleaned is determined based on the hash value. Data to be cleaned that has a hash Hamming distance less than a preset Hamming distance threshold is identified as the first duplicate data. Obtain the remaining data to be cleaned from each set of data to be cleaned, excluding the first duplicate data. Based on the pixel values ​​of the remaining data to be cleaned, determine the pixel similarity between every two remaining data to be cleaned. The remaining data to be cleaned with pixel similarity greater than a preset pixel similarity threshold are identified as the second duplicate data; The first duplicate data and the second duplicate data are deduplicated.

9. The method according to claim 6, characterized in that, The method further includes: The cleaning effect of the low-quality cleaning task is verified to obtain the first verification result. If the first verification result does not meet the preset first verification condition, the low-quality cleaning parameters of the low-quality cleaning task are adjusted according to the first verification result. And / or, perform cleaning effect verification processing on the deduplication cleaning task to obtain a second verification result; If the second verification result does not meet the preset second verification condition, the deduplication cleaning parameters of the deduplication cleaning task are adjusted according to the second verification result.

10. The method according to claim 9, characterized in that, The step of performing a cleaning effect verification process on the low-quality cleaning task to obtain a first verification result includes: Negative data injection processing is performed on the pre-cleaning data corresponding to the low-quality cleaning task to obtain a low-quality cleaning verification dataset. The low-quality cleaning task and recall rate calculation are performed on the low-quality cleaning and verification dataset to obtain the first verification result.

11. The method according to claim 9, characterized in that, The step of performing a cleaning effect verification process on the deduplication cleaning task to obtain a second verification result includes: Negative data injection processing is performed on the pre-cleaning data corresponding to the deduplication and cleaning task to obtain the deduplication and cleaning verification dataset. The deduplication and cleanup task and recall calculation are performed on the deduplication and cleanup verification dataset to obtain the second verification result.

12. The method according to claim 9, characterized in that, The step of performing a cleaning effect verification process on the deduplication cleaning task to obtain a second verification result includes: The cleaned data corresponding to the deduplication and cleaning task is sampled to obtain deduplication and cleaning sampled data. Obtain the similarity matrix between every two deduplicated and cleaned sampled data; Based on the similarity of a specified region in the similarity matrix, deduplicated and cleaned similar data are determined when the similarity of the specified region is greater than a preset feature similarity threshold. The similarity ratio of deduplication and cleaning is determined based on the number of deduplication and cleaning sample data and the number of deduplication and cleaning similar data. The second verification result is determined based on the comparison between the deduplication and cleaning similarity ratio and the preset deduplication and cleaning similarity ratio threshold.

13. The method according to any one of claims 1 to 12, characterized in that, After performing the cleaning task on the dataset to be cleaned and obtaining the data cleaning result, the method further includes: The cleaned dataset in the data cleaning results is subjected to quality assessment processing to obtain a quality analysis report; Based on the quality analysis report, determine whether to perform data optimization processing on the cleaned dataset.

14. The method according to claim 13, characterized in that, The step of determining whether to perform data optimization processing on the cleaned dataset based on the quality analysis report includes: The quality score of each cleaned data in the cleaned dataset is determined based on the quality analysis report. Based on the quality score, determine the data to be optimized in each cleaned data set; The data to be optimized is subjected to image enhancement and optimization processing.

15. The method according to claim 13, characterized in that, The step of determining whether to perform data optimization processing on the cleaned dataset based on the quality analysis report includes: The number of samples corresponding to each data type in the cleaned dataset is determined based on the quality analysis report. For data types with fewer than a preset sample number threshold, data supplementation and optimization processing is performed.

16. The method according to claim 1, characterized in that, After performing the cleaning task on the dataset to be cleaned and obtaining the data cleaning result, the method further includes: The cleaning effectiveness of the cleaned dataset in the data cleaning results is verified according to the preset model training task to obtain the cleaning effectiveness of the cleaned dataset. If the cleaning effectiveness does not meet the preset cleaning effectiveness conditions, then the cleaning parameters of the target cleaning strategy are adjusted.

17. The method according to claim 16, characterized in that, The method further includes: Based on preset confidence thresholds and loss thresholds, identify difficult examples in the cleaned dataset when performing the preset model training task; The cleaning parameters of the target cleaning strategy are adjusted based on the image features of the difficult sample. The key difficult examples containing key objectives are marked and retained.

18. An electronic device, characterized in that, The method includes a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the method according to any one of claims 1 to 17.

19. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the method described in any one of claims 1 to 17.

Images