Optimization processing method applied to sample data set of image recognition

Abstract

This invention discloses a method for centralized optimization of sample datasets applied to image recognition, relating to the field of sample image data processing technology. The invention first extracts visual features from erroneous sample data, obtains clusters using a clustering algorithm, and analyzes cluster attributes to determine the type of missing samples. Then, it calls the associated model of the corresponding model task, matches the sample quality dimensions corresponding to the model's anomaly indicators, and filters out the target quality dimensions with anomalies through pre-trained models or manual verification of the sample dataset. Finally, it optimizes the sample dataset of the target model according to the logic of first optimizing quality and then optimizing quantity. This invention overcomes the limitations of traditional static dataset processing that relies on experience to judge data requirements. On the one hand, it accurately locates gap scenarios not covered by the model through clustering, avoiding sample redundancy or insufficient samples in key scenarios. On the other hand, it combines the target quality dimensions to remove low-quality data, ensuring that the supplemented samples meet both quantity and quality requirements.

Description

Technical Field

[0001] This invention belongs to the field of sample image data processing technology, specifically a method for centralized optimization processing of sample data applied to image recognition. Background Technology

[0002] In the field of computer vision, the performance of image recognition models is highly dependent on the size, diversity, and annotation quality of the sample dataset. Existing sample dataset processing techniques mostly adopt a static centralized processing mode. The core process is as follows: after completing the dataset construction through multi-source data integration, cleaning and deduplication, and manual or semi-automatic annotation, it is fixedly divided into training set, validation set, and test set. The dataset content is no longer dynamically adjusted according to model feedback during subsequent model training.

[0003] Current technologies generally rely on human experience to judge whether sample data is sufficient, failing to quantify the actual data gaps in the model. This leads to either data redundancy due to over-supplementation of common scenario samples, consuming significant storage and computing resources, or significant degradation in generalization ability in extreme scenario samples due to omissions. Moreover, current technologies mostly focus on basic quality checks before data output, failing to determine whether high-quality data is suitable for model training needs. If data quality dimensions and model anomaly indicators do not match, the problem of high data quality but poor model performance can still occur.

[0004] This invention provides a method for centralized optimization of sample data for image recognition, in order to solve the above-mentioned technical problems. Summary of the Invention

[0005] This invention aims to at least solve one of the technical problems existing in the prior art. To this end, this invention proposes a centralized optimization processing method for sample datasets applied to image recognition. This method breaks through the limitations of traditional static dataset processing that relies on experience to judge data requirements. On the one hand, it accurately locates gap scenes not covered by the model through clustering, avoiding the problems of sample redundancy or insufficient samples in key scenes. On the other hand, it combines the target quality dimension to remove low-quality data, ensuring that the supplemented samples meet both quantity and quality requirements. The optimized sample dataset can accurately match the model training requirements, significantly improve the model's generalization ability, reduce the cost of processing invalid data, avoid redundant data occupying computing power, shorten the model iteration cycle, and make the model's recognition performance more stable in complex scenes.

[0006] To achieve the above objectives, a first aspect of the present invention provides a method for centralized optimization processing of sample data applied to image recognition, comprising:

[0007] Obtain error sample data and model anomaly indicators for the target model; where error sample data includes sample data where the prediction results are inconsistent with the sample labels and error types;

[0008] Error sample data is analyzed using cluster analysis to determine the types of missing samples; the target quality dimension corresponding to the model anomaly index is determined based on the association model; where the target quality dimension is the quality dimension of the anomaly samples.

[0009] The sample dataset for optimizing the target model based on the gap sample type and the target quality dimension.

[0010] In one feasible approach, error sample data is analyzed using clustering analysis methods, including:

[0011] Extract visual features from erroneous sample data; perform cluster analysis on the visual features using a clustering algorithm to obtain several clusters;

[0012] The cluster attributes of several clusters are analyzed and determined, and the cluster attributes are converted into gap sample types; among them, the cluster attributes include target type, scene type, target state and shooting conditions.

[0013] In one feasible approach, the target quality dimension corresponding to the model anomaly index is determined based on the correlation model, including:

[0014] Extract the associated models of the target model and the corresponding model tasks; where the model tasks include image classification tasks, object detection tasks, and semantic segmentation tasks.

[0015] The sample quality dimension corresponding to the abnormal indicators of the model is matched by the correlation model; the sample data is analyzed based on the matched sample quality dimension to determine the target quality dimension.

[0016] In one feasible approach, the construction of the association model includes:

[0017] Define the model metrics and sample quality dimensions for the model task;

[0018] Extract historical datasets for the model task, evaluate the correlation between model metrics and sample quality dimensions based on the historical datasets, and construct a correlation model between model metrics and sample quality dimensions based on the correlation.

[0019] In one feasible approach, the correlation between model metrics and sample quality dimensions is evaluated based on historical datasets, including:

[0020] Calculate the sample dimension quality of the historical dataset and extract model metrics when training the model using the historical dataset;

[0021] The correlation analysis method is used to analyze the correlation between sample dimensional quality and model indicators; among which, the correlation analysis method includes Pearson correlation analysis or Spearman correlation analysis.

[0022] In one feasible approach, a correlation model is constructed based on the correlation between model metrics and sample quality dimensions, including:

[0023] Set a threshold for the correlation coefficient;

[0024] When the correlation coefficient between the sample quality dimension and the model metric is greater than the coefficient threshold, the sample quality dimension and the model metric are associated.

[0025] A correlation model is constructed based on several model indicators and their associated sample quality dimensions.

[0026] In one feasible approach, sample data is analyzed based on the matched sample quality dimension, including:

[0027] The quality dimensions of the matched samples are sequentially labeled as the basic quality dimensions;

[0028] Extract the sample dataset for training the target model, and analyze whether there are any anomalies in the sample dataset based on the basic quality dimension; if so, mark the basic quality dimension as the target quality dimension.

[0029] In one feasible approach, the sample dataset for optimizing the target model is based on the gap sample type and the target quality dimension, including:

[0030] The sample problem is determined based on the gap sample type and the target quality dimension; the sample problem includes the sample quantity problem and the sample quality problem.

[0031] Optimize the sample dataset based on the sample problem.

[0032] In one feasible approach, the sample dataset is optimized based on the sample problem, including:

[0033] Extract the sample dataset of the target model;

[0034] Based on the issue of sample quality, the sample data in the sample dataset is optimized for quality; based on the issue of sample quantity, the optimized sample dataset is optimized for quantity.

[0035] In one feasible approach, the quantity optimization of the quality-optimized sample dataset is performed according to the sample size problem, including:

[0036] Extract the standard distribution of the sample dataset; where the standard distribution includes the data types of the samples when all model metrics are normal and the sample ratios between each data type.

[0037] Based on the issue of sample quantity, the amount of supplementary samples of each data type in the sample dataset after quality optimization is statistically analyzed; and the sample dataset is supplemented and optimized based on the amount of supplementary samples.

[0038] Compared with the prior art, the beneficial effects of the present invention are:

[0039] 1. This invention first acquires erroneous sample data and model anomaly indicators of the target model; by extracting visual features from the erroneous sample data, clusters are obtained through clustering algorithms, and the cluster attributes are analyzed to determine the gap sample type; then, the associated model of the corresponding model task is called to match the sample quality dimension corresponding to the model anomaly indicator, and the target quality dimension with anomalies is screened out through pre-trained models or manual verification of sample datasets; finally, the sample dataset of the target model is optimized according to the logic of first optimizing quality and then optimizing quantity. This invention breaks through the limitations of traditional static dataset processing that relies on experience to judge data requirements. On the one hand, it accurately locates gap scenarios not covered by the model through clustering, avoiding the problems of sample redundancy or insufficient samples in key scenarios; on the other hand, it combines the target quality dimension to remove low-quality data, ensuring that the supplemented samples meet both quantity and quality requirements. The optimized sample dataset can accurately match the model training requirements, significantly improve the model's generalization ability, reduce the cost of invalid data processing, avoid redundant data occupying computing power, shorten the model iteration cycle, and make the model's recognition performance more stable in complex scenarios.

[0040] 2. In constructing the associated model, this invention first defines specific model metrics and sample quality dimensions for different model tasks such as image classification, object detection, and semantic segmentation. Then, it collects multiple versions of historical datasets, calculates the sample quality dimensions and corresponding model metrics for each version, quantifies the correlation between the two through correlation analysis, sets coefficient thresholds to filter strongly correlated sample quality dimensions, and constructs a mapping rule between abnormal model metrics and sample quality dimensions. This invention addresses the pain point of traditional model training that only verifies data output quality without judging model suitability. By quantifying the correlation through historical datasets, it transforms quality optimization from subjective judgment to data-driven approaches. Simultaneously, the associated model adapts to multiple task scenarios, providing targeted quality optimization directions for different tasks, improving the accuracy of data quality problem localization, reducing manual investigation costs, ensuring that core model metrics remain at a stable high level, and improving the efficiency and reliability of model training. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic diagram of the method steps of the sample data central optimization processing method in an embodiment of the present invention;

[0043] Figure 2This is a schematic diagram illustrating the construction process of the association model in an embodiment of the present invention. Detailed Implementation

[0044] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] Please see Figure 1 The first aspect of the present invention provides a method for centralized optimization processing of sample data applied to image recognition, comprising:

[0046] Obtain error sample data and model anomaly indicators for the target model; where error sample data includes sample data where the prediction results are inconsistent with the sample labels and error types;

[0047] Error sample data is analyzed using cluster analysis to determine the types of missing samples; the target quality dimension corresponding to the model anomaly index is determined based on the association model; where the target quality dimension is the quality dimension of the anomaly samples.

[0048] The sample dataset for optimizing the target model based on the gap sample type and the target quality dimension.

[0049] A target model is an artificial intelligence model trained on a sample dataset to perform a specific model task. When training a target model using sample images from the sample dataset, if the number of sample images is insufficient, the trained target model will fail to meet the set requirements for recognizing certain scenes. If the number is excessive, the target model will need to process a large amount of redundant data during training, affecting training efficiency. If the quality of the sample images is insufficient or unsuitable, the overall performance of the trained target model will fail to meet the set requirements.

[0050] This embodiment first acquires erroneous sample data and model anomaly indicators during the training of the target model. Based on the erroneous sample data, it determines which sample data the target model cannot accurately identify, thus determining the types of gap samples required by the target model. Based on the model anomaly indicators, it identifies which core metrics of the target model are abnormal, and then determines which sample quality dimensions are abnormal based on these abnormal core metrics, ultimately determining the required sample data. Finally, the gap sample types and sample quality dimensions are combined to optimize the sample dataset of the target model, ensuring that the optimized sample dataset meets the requirements in both quantity and quality.

[0051] In order to address the issues of quantity and quality of sample data during the training of artificial intelligence models, this embodiment can simultaneously acquire erroneous sample data and model anomaly indicators of several target models. After analyzing the sample data that needs to be optimized for each target model, the data is centrally summarized. Based on the summary results, the sample data is centrally optimized and redistributed, which can improve optimization efficiency and ensure that the issues of quantity and quality of samples are resolved.

[0052] In a preferred embodiment, the error sample data is analyzed by cluster analysis, including: extracting visual features of the error sample data; performing cluster analysis on the visual features using a clustering algorithm to obtain several clusters; analyzing and determining the cluster attributes of the several clusters, and converting the cluster attributes into gap sample types; wherein, the cluster attributes include target type, scene type, target state, and shooting conditions.

[0053] By embedding an error sample capture module during model training, an error sample hook can be inserted into the model training framework. When the model's prediction result for the sample data is inconsistent with the sample label, the error sample recording mechanism is automatically triggered to record the error sample in real time and mark the error type for the error sample. Error types include classification error, localization error, missed detection / false detection, etc.

[0054] Visual features of captured erroneous samples are extracted using a pre-trained feature extraction module, and these visual features are then associated with the erroneous sample data. A clustering algorithm is then used to perform cluster analysis on the visual features of the erroneous samples, and the type of missing sample is identified by analyzing the cluster attributes of each cluster (such as target type, scene type, target state, shooting conditions, etc.).

[0055] As an example, and not a limitation, let's take the training of a vehicle target detection model for urban roads (based on YOLOv8) as an example:

[0056] In the validation phase of the YOLOv8 training framework (evaluation on the validation set after each training round), an error sample hook is inserted. The hook function is bound to the stage after the model prediction output and before the loss calculation, and the model prediction results are compared with the sample labels in real time to obtain the error sample data.

[0057] For the sample data in the erroneous sample data, a pre-trained ResNet-50 model is used to analyze the sample data, outputting feature vectors containing information such as target appearance, scene lighting, and background environment as visual features. The K-Means clustering algorithm is then used to perform cluster analysis on the visual features of all sample data in the erroneous sample data, with each cluster either manually labeled or automatically having its cluster attributes extracted.

[0058] For example, one cluster might have the following cluster attributes: Target type: sedan; Scene type: rainy night (streetlights + wiper reflection); Target state: partially obscured by reflected water; Shooting conditions: low light (ISO > 800), image blur (motion blur). To resolve this clustering issue, the required sample data needs to include sedans in rainy night conditions with reflected water. Therefore, the missing sample type can be defined as: rainy night conditions with reflected water. Supplementing the sample data based on this missing sample type can resolve the clustering problem.

[0059] The current centralized optimization process for sample data is static; once processed, the sample data is fixed in the sample dataset. This leads to the sample dataset being unable to adapt to the dynamic needs of model training. This embodiment records error sample logs during target model training, classifies error samples using clustering algorithms, identifies gap sample types, and dynamically supplements sample data based on these gap sample types. The supplemented sample data is then processed and integrated into the original dataset. Using the new dataset for the target model does not affect the model's iteration efficiency. This solves the problem of traditional centralized optimization processes relying on experience to judge whether sample data is sufficient, failing to quantify data gaps in the model, and leading to either data redundancy or insufficient scenario samples.

[0060] In a preferred embodiment, determining the target quality dimension corresponding to the model anomaly index based on the association model includes: extracting the association model of the model task corresponding to the target model; wherein the model task includes image classification task, object detection task and semantic segmentation task; matching the sample quality dimension corresponding to the model anomaly index through the association model; and analyzing the sample data based on the matched sample quality dimension to determine the target quality dimension.

[0061] During model training, a correlation model between model metrics and sample quality dimensions is established. If a model metric declines, the corresponding dimension of the sample data is automatically traced back. Once it is determined that there is a problem with the quality of the corresponding dimension, it is optimized in a targeted manner to ensure that all model metrics are at a high level.

[0062] This technical solution can solve the problem that traditional model training only performs quality checks before data output, which cannot determine whether the data quality is suitable for the model. This may result in high data labeling accuracy, but the trained model still performs poorly.

[0063] In a preferred embodiment, please refer to Figure 2 The construction of the correlation model includes: defining the model metrics and sample quality dimensions of the model task; extracting the historical dataset of the model task and evaluating the correlation between the model metrics and sample quality dimensions based on the historical dataset; and constructing a correlation model between the model metrics and sample quality dimensions based on the correlation.

[0064] To address the issue of high-quality data but poor model performance, it is necessary to first define the model metrics for different tasks, then establish a quantitative correlation between the model metrics and sample quality dimensions, and finally locate the problems in the sample data through the correlation model in order to solve the sample problems in a targeted manner.

[0065] Model metrics need to be defined in conjunction with the specific task; different tasks exhibit different characteristics of "poor performance," corresponding to varying quality issues in the sample data. The following are some examples of model metrics for various tasks:

[0066] 1. Core metrics for image classification tasks:

[0067] 1) Accuracy: The percentage of samples that are correctly predicted overall; applicable to balanced datasets.

[0068] 2) Precision: The proportion of sample data that are predicted to be of a certain class but actually belong to that class, evaluating the problem of misclassification;

[0069] 3) Recall: In fact, it is the proportion of samples of a certain class that are correctly predicted, and it evaluates the problem of missed detections;

[0070] 4) F1 score: The harmonic mean of precision and recall, used to evaluate the predictive performance of a category;

[0071] 5) Confusion matrix: The confusion situation of specific categories, such as a cat being mistakenly identified as a dog.

[0072] 2. Core metrics for object detection tasks:

[0073] 1) mAP: Average accuracy across all categories, used to evaluate detection effectiveness;

[0074] 2) False negative rate: The proportion of actual targets that are not detected;

[0075] 3) False detection rate: The proportion of detected targets that do not actually exist;

[0076] 4) Positioning error: The degree of offset between the predicted bounding box and the ground truth bounding box (e.g., IOU value).

[0077] 3. Core metrics for semantic segmentation tasks:

[0078] 1) MIoU: Mean Intersection over Union (IoU) for all categories;

[0079] 2) Pixel accuracy: The proportion of all pixels that are correctly classified;

[0080] 3) Category accuracy: The proportion of pixels that are correctly classified in a certain category.

[0081] Similarly, the sample quality dimensions differ for different model tasks. Taking the object detection task as an example, the sample quality dimensions are defined as shown in Table 1:

[0082] Table 1. Sample quality dimensions and their impact model metrics for object detection tasks.

[0083] Sample quality dimension Definitions Potentially affected model metrics Labeling accuracy The degree of matching between the true bounding box and the label bounding box mAP, positioning error, false detection rate Category balance Distribution of sample size by category Recall rate, mAP Scene diversity Coverage of the same category of sample data in different scenarios (background, angle, lighting, etc.) mAP Target integrity Is the target obscured / trunculated? False negative rate Image clarity Resolution / Blur of the target area Small goal mAP

[0084] Next, taking object detection as an example, we will construct a correlation model between model metrics and sample quality dimensions:

[0085] First, collect historical datasets of different versions for the object detection task. For each historical dataset, calculate the sample quality dimension and extract the model metrics for the object detection model corresponding to each version of the historical dataset. Next, calculate the correlation coefficient between the sample quality dimension and the model metrics through correlation analysis; the larger the absolute value of the correlation coefficient, the stronger the correlation between the two, with a positive correlation coefficient indicating a positive correlation and a negative correlation coefficient indicating a negative correlation.

[0086] An interpretable correlation model is constructed based on the correlation between model metrics and sample quality dimensions. That is, based on the above analysis, a mapping rule is generated between abnormal model metrics and possible sample quality dimensions.

[0087] In a preferred embodiment, evaluating the correlation between model metrics and sample quality dimensions based on historical datasets includes: calculating the sample dimension quality of the historical dataset and extracting model metrics when training the model using the historical dataset; analyzing the correlation between sample dimension quality and model metrics based on correlation analysis methods; wherein the correlation analysis methods include Pearson correlation analysis or Spearman correlation analysis.

[0088] For example, the sample quality dimensions and corresponding model metrics for the four versions of the historical dataset are shown in Table 2:

[0089] Table 2. Sample quality dimensions of historical datasets and corresponding model metrics

[0090] Historical dataset versions Sample quality dimension Model Indicators V1.0 Annotation accuracy = 0.80, Class balance = 0.60, Scene diversity = 1.0, Target completeness = 0.65, Image sharpness = 6.0 mAP=0.62, false negative rate=0.30, false positive rate=0.25, positioning error=0.35, small target mAP=0.45 V2.0 Annotation accuracy = 0.92, class balance = 0.85, scene diversity = 1.1, target integrity = 0.68, image sharpness = 6.2 mAP=0.75, false negative rate=0.22, false positive rate=0.12, positioning error=0.20, small target mAP=0.48 V3.0 Annotation accuracy = 0.93, class balance = 0.88, scene diversity = 1.5, target integrity = 0.82, image sharpness = 7.8 mAP=0.83, false negative rate=0.15, false positive rate=0.08, positioning error=0.18, small target mAP=0.65 V4.0 Annotation accuracy = 0.92, class balance = 0.86, scene diversity = 1.1, target integrity = 0.80, image sharpness = 6.5 mAP=0.76, false negative rate=0.17, false positive rate=0.09, positioning error=0.19, small target mAP=0.52

[0091] The correlation between sample dimensional quality and model metrics, calculated using the Pearson correlation coefficient, is shown in Table 3.

[0092] Table 3. Correlation between sample dimensional quality and model metrics

[0093] Model Indicators Correlation coefficient with annotation accuracy Correlation coefficient with class balance Correlation coefficient with scene diversity Correlation coefficient with target integrity Correlation coefficient with image sharpness mAP 0.91 (strong positive correlation) 0.87 (strong positive correlation) 0.89 (strong positive correlation) 0.72 (moderate positive correlation) 0.68 (moderate positive correlation) False negative rate -0.65 (moderate negative correlation) -0.85 (strong negative correlation) -0.70 (moderate negative correlation) -0.92 (strong negative correlation) -0.55 (weak negative correlation) False positive rate -0.93 (strong negative correlation) -0.60 (moderate negative correlation) -0.52 (weak negative correlation) -0.48 (weak negative correlation) -0.45 (weak negative correlation) Positioning error -0.90 (strong negative correlation) -0.50 (weak negative correlation) -0.55 (weak negative correlation) -0.42 (weak negative correlation) -0.58 (moderate negative correlation) Small goal mAP 0.62 (moderate positive correlation) 0.58 (moderate positive correlation) 0.65 (moderate positive correlation) 0.52 (weak positive correlation) 0.90 (strong positive correlation)

[0094] It should be noted that historical datasets can be sample datasets corresponding to previous versions of the target model, or sample datasets used for training other models with similar model tasks.

[0095] In a preferred embodiment, constructing a correlation model between model indicators and sample quality dimensions based on correlation includes: setting a correlation coefficient threshold; associating the sample quality dimension and model indicators when the correlation coefficient between the sample quality dimension and the model indicators is greater than the coefficient threshold; and constructing a correlation model based on several model indicators and their associated sample quality dimensions.

[0096] As shown in Table 3, different correlation coefficient thresholds can be set for different model anomaly indicators. For example, a threshold of moderate correlation or higher can be set, or a threshold of 0.55 can be set. The established association models are shown in Table 4 below.

[0097] Table 4. Correlation Model between Model Indicators and Sample Quality Dimensions

[0098] Model anomaly indicators Possible dimensions of sample quality Mapping basis mAP decreased significantly ① Decreased scene diversity (e.g., missing key scene samples, leading to poor model generalization); ② Decreased annotation accuracy (incorrect annotations cause model learning bias); ③ Decreased class balance (a sharp decrease in samples of a certain class lowers overall accuracy). Scene diversity was strongly positively correlated with mAP (0.89); annotation accuracy was strongly positively correlated with mAP (0.91); class balance was strongly positively correlated with mAP (0.87). The false negative rate suddenly increased ① Decreased target integrity (a large number of newly added occluded / truncated samples, making it difficult for the model to identify); ② Decreased class balance (more missed detections in classes with fewer samples). Target completeness was strongly negatively correlated with the false negative rate (-0.92); class balance was strongly negatively correlated with the false negative rate (-0.85). The false positive rate has increased significantly. ① Decreased labeling accuracy (incorrect label boxes or false positive labels, the model learns incorrect target features) Labeling accuracy is strongly negatively correlated with false detection rate (-0.93) (the only strongly correlated dimension). Increased positioning error ① Decreased annotation accuracy (annotation box offset, model learns incorrect localization patterns); ② Decreased image sharpness (blurred targets cause predicted box offset). Labeling accuracy is strongly negatively correlated with positioning error (-0.90); image sharpness is moderately negatively correlated with positioning error (-0.58). The small target mAP remains low. ① Decreased image clarity (small targets have fewer pixels, and blurring increases the difficulty of detection); ② Insufficient annotation accuracy (the bounding boxes of small targets are prone to shifting, affecting learning). Image sharpness was strongly positively correlated with small target mAP (0.90); annotation accuracy was moderately positively correlated with small target mAP (0.62).

[0099] In a preferred embodiment, analyzing sample data based on matched sample quality dimensions includes: sequentially labeling the matched sample quality dimensions as basic quality dimensions; extracting the sample dataset for training the target model, and analyzing whether there are anomalies in the sample dataset based on the basic quality dimensions; if yes, labeling the basic quality dimensions as the target quality dimensions.

[0100] After matching the sample quality dimension associated with the model anomaly index through the associated model corresponding to the target model, it does not mean that the sample dataset has problems corresponding to all matching sample quality dimensions. It is possible that the sample dataset only has problems in a certain sample quality dimension.

[0101] The matched sample quality dimensions are sequentially labeled as basic quality dimensions. The sample dataset of the target model is analyzed by pre-training model or by manual analysis to see if there is a problem corresponding to the basic quality dimension. If so, the basic quality dimension is labeled as the target quality dimension.

[0102] For example, mAP decreased significantly, and the sample quality dimensions matched by the association model included decreased scene diversity, decreased annotation accuracy, and decreased class balance.

[0103] To address the diversity of scenarios, the sample data in the sample dataset is automatically labeled, and the proportion of each scenario is calculated. If the proportion of a certain scenario changes abnormally relative to the standard dataset, it is determined that the sample dataset has a scenario diversity problem.

[0104] To address the labeling accuracy, sample data is randomly selected from the sample dataset, and the labels of the selected sample data are manually reviewed. If the reviewed labeling accuracy is consistent with that of the standard dataset, then there is no labeling accuracy issue in the sample dataset.

[0105] To address class balance, the ratio of the number of samples from the least popular class to the number of samples from the most popular class in the sample dataset is calculated. If this ratio is consistent with that of the standard dataset, then the sample dataset does not have a class balance problem.

[0106] It is worth noting that the standard dataset can be a historical dataset of a certain version. For example, in the pedestrian and non-motorized vehicle target detection task of intelligent traffic cameras, "mAP decreased significantly, specifically from 0.83 in V3.0 to 0.72 in V4.0, a decrease of 13%", then the historical dataset of version V3.0 can be used as the standard dataset. Of course, the standard dataset can also be a pre-adjusted dataset that meets various requirements for model training.

[0107] In a preferred embodiment, optimizing the sample dataset of the target model based on the gap sample type and the target quality dimension includes: determining the sample problem based on the gap sample type and the target quality dimension; wherein the sample problem includes the sample quantity problem and the sample quality problem; and optimizing the sample dataset based on the sample problem.

[0108] In a preferred embodiment, optimizing the sample dataset according to the sample problem includes: extracting the sample dataset of the target model; optimizing the sample data in the sample dataset according to the sample quality problem; and optimizing the quantity of the quality-optimized sample dataset according to the sample quantity problem.

[0109] After determining the types of gapped samples and the target quality dimension in the sample dataset, it is necessary to comprehensively identify the sample problems existing in the dataset. If there are problems with the target quality dimension of the sample dataset, the sample data in the dataset is optimized according to the target quality dimension; for example, if the annotation in the sample data is inaccurate, the sample data is re-annotated; if the image clarity of the sample data is insufficient, the sample data is removed. It can be seen that in the quality optimization process, it may only be necessary to optimize the sample data, or it may be necessary to supplement the sample data.

[0110] After quality optimization, the issue of sample quantity in the dataset needs to be addressed. As mentioned earlier, after quality optimization, it is necessary not only to replenish the missing sample types but also to replenish the sample data removed after quality optimization, in order to compensate for the reduction in sample data caused by the optimization.

[0111] In a preferred embodiment, the quantity optimization of the quality-optimized sample dataset is performed according to the sample quantity problem, including: extracting the standard distribution of the sample dataset; wherein, the standard distribution of the sample dataset includes the sample data types when all model indicators are normal and the sample ratios between each sample data type; calculating the sample supplementation amount of each sample data type in the quality-optimized sample dataset based on the sample quantity problem; and supplementing and optimizing the sample dataset based on the sample supplementation amount.

[0112] Optimizing the quantity of the sample dataset essentially involves supplementing it to ensure sufficient and balanced sample data. After quality optimization, the image attributes of the removed sample data are statistically analyzed, and the content of these image attributes is consistent with the cluster attributes. The statistical results are used to determine the amount of sample data to be supplemented. This embodiment uses the standard distribution of the sample dataset as a benchmark. If the sample dataset needs to be supplemented with a certain type of sample data, the amount of sample data to be supplemented is determined based on the existing quantity of that type of sample data and its proportion compared to other types of sample data. It is worth noting that the standard distribution of the sample dataset can be obtained from a statistical standard dataset.

[0113] For example, suppose the problem of determining the number of samples based on the gap sample type is: to supplement image data including cars reflecting light in puddles during a rainstorm night. During quality optimization, exactly 50 images of cars reflecting light in puddles during a rainstorm night are deemed unqualified and are removed. This leaves 950 images of cars reflecting light in puddles during a rainstorm night in the sample dataset. Proportionally, there should be 1500 images, so 550 more images of cars reflecting light in puddles during a rainstorm night need to be supplemented.

[0114] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A method for centralized optimization processing of sample data applied to image recognition, characterized in that, include: Obtain error sample data and model anomaly indicators for the target model; where error sample data includes sample data where the prediction results are inconsistent with the sample labels and error types; The erroneous sample data is analyzed using clustering analysis to determine the gap sample type; the target quality dimension corresponding to the model anomaly index is determined based on the association model; wherein, the target quality dimension is the quality dimension of the anomalous samples. The sample dataset for optimizing the target model is based on the gap sample type and the target quality dimension; The erroneous sample data was analyzed using cluster analysis methods, including: Visual features of the erroneous sample data are extracted; clustering analysis is performed on the visual features using a clustering algorithm to obtain several clusters; The cluster attributes of several clusters are analyzed and determined, and the cluster attributes are converted into gap sample types; wherein, the cluster attributes include target type, scene type, target state and shooting conditions; The target quality dimension corresponding to the abnormal indicators of the model is determined based on the correlation model, including: Extract the associated models of the model tasks corresponding to the target model; wherein, the model tasks include image classification tasks, object detection tasks, and semantic segmentation tasks; The sample quality dimension corresponding to the abnormal indicator of the model is matched by the association model; the sample data is analyzed based on the matched sample quality dimension to determine the target quality dimension; The construction of the association model includes: Define the model metrics and sample quality dimensions for the model task; among which, the sample quality dimension includes annotation accuracy, class balance, scene diversity, target integrity and image sharpness. The model metrics corresponding to standard accuracy include mAP, localization error and false detection rate. The model metrics corresponding to class balance include recall and mAP. The model metric corresponding to scene diversity is mAP. The model metric corresponding to target integrity is false negative rate. The model metric corresponding to image sharpness is small target mAP. Extract the historical dataset of the model task, evaluate the correlation between model metrics and sample quality dimensions based on the historical dataset, and construct a correlation model between model metrics and sample quality dimensions based on the correlation.

2. The sample data central optimization processing method for image recognition according to claim 1, characterized in that, The correlation between model metrics and sample quality dimensions is evaluated based on the historical dataset, including: Calculate the sample dimension quality of the historical dataset and extract the model metrics when training the model using the historical dataset; The correlation between the sample dimensional quality and the model indicators is analyzed based on correlation analysis methods; wherein, the correlation analysis methods include Pearson correlation analysis or Spearman correlation analysis.

3. The sample data central optimization processing method for image recognition according to claim 2, characterized in that, Based on the correlation, a correlation model is constructed between model metrics and sample quality dimensions, including: Set a threshold for the correlation coefficient; When the correlation coefficient between the sample quality dimension and the model indicator is greater than the coefficient threshold, the sample quality dimension and the model indicator are associated. A correlation model is constructed based on several model metrics and their associated sample quality dimensions.

4. The sample data central optimization processing method for image recognition according to claim 1, characterized in that, Analyzing sample data based on the matched sample quality dimensions includes: The quality dimensions of the matched samples are sequentially labeled as basic quality dimensions; Extract the sample dataset used to train the target model, and analyze whether there are any anomalies in the sample dataset based on the basic quality dimension; if so, mark the basic quality dimension as the target quality dimension.

5. The sample data central optimization processing method for image recognition according to any one of claims 1 to 4, characterized in that, The sample dataset for optimizing the target model based on the gap sample type and the target quality dimension includes: The sample problem is determined based on the gap sample type and the target quality dimension; wherein, the sample problem includes the sample quantity problem and the sample quality problem; Optimize the sample dataset based on the sample problem.

6. The sample data central optimization processing method for image recognition according to claim 5, characterized in that, Optimize the sample dataset based on the sample problem, including: Extract the sample dataset of the target model; The quality of the sample data in the sample dataset is optimized according to the sample quality problem; the quantity of the optimized sample dataset is optimized according to the sample quantity problem.

7. The sample data central optimization processing method for image recognition according to claim 6, characterized in that, The quantity optimization of the quality-optimized sample dataset is performed according to the aforementioned sample quantity problem, including: Extract the standard distribution of the sample dataset; wherein, the standard distribution of the sample dataset includes the data types of the samples when all model indicators are normal and the sample ratios between each data type; Based on the issue of sample quantity, the amount of supplementary samples for each data type in the sample dataset after quality optimization is statistically analyzed; and the sample dataset is further optimized based on the amount of supplementary samples.

Images