Sample data centralized optimization processing method applied to image recognition

Abstract

The invention discloses a sample data centralized optimization processing method applied to image recognition, and relates to the technical field of sample image data processing. According to the method, visual features of error sample data are extracted, clusters are obtained through a clustering algorithm, and cluster attributes are analyzed to determine gap sample types; calling an association model corresponding to the model task, matching a sample quality dimension corresponding to a model abnormal index, and screening out an abnormal target quality dimension through a pre-training model or manual verification of a sample data set; finally, a sample data set of the target model is optimized according to the logic that quality optimization is conducted firstly and then quantity optimization is conducted; the method breaks through the limitation that data requirements are judged according to experience in traditional static data set processing, on one hand, gap scenes which are not covered by a model are accurately positioned through clustering, and the problem that samples are redundant or key scene samples are insufficient is avoided; and on the other hand, low-quality data is removed in combination with the target quality dimension, and it is ensured that the supplemented samples have both the number standard and the quality adaptation.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of sample image data processing, and specifically relates to a sample data set optimization processing method applied to image recognition. BACKGROUND

[0002] In the field of computer vision, the performance of an image recognition model is highly dependent on the size, diversity and annotation quality of a sample data set. Existing sample data set processing techniques mostly adopt a static centralized processing mode, and the core process is as follows: after a data set is constructed through multi-source data integration, cleaning and deduplication, and manual or semi-automatic annotation, the data set is fixedly divided into a training set, a validation set and a test set, and the content of the data set is no longer dynamically adjusted according to model feedback in the subsequent model training process.

[0003] Existing techniques generally rely on manual experience to determine whether sample data is sufficient, and cannot quantify the actual data gap of a model. Either too many samples of a common scene are supplemented, resulting in data redundancy and occupying a large amount of storage and computing resources, or extreme scene samples are missed, resulting in a significant decrease in the generalization ability of the model in such a scene. Moreover, existing techniques mostly focus on basic quality inspection before data output, and cannot determine whether high-quality data is suitable for model training requirements. If the quality dimensions of the data and the abnormal indicators of the model do not match, the problem of high data quality but poor model effect will still occur.

[0004] The application provides a sample data set optimization processing method applied to image recognition to solve the above technical problems. SUMMARY

[0005] The application aims to at least solve one of the technical problems in the prior art. To this end, the application provides a sample data set optimization processing method applied to image recognition. This method breaks through the limitation of traditional static data set processing that relies on experience to determine data requirements. On the one hand, it accurately locates the gap scene that is not covered by the model through clustering, avoiding the problems of sample redundancy or insufficient key scene samples. On the other hand, it removes low-quality data in combination with target quality dimensions, ensuring that the supplemented samples meet the quantity requirement and are suitable in quality. The optimized sample data set can accurately match the model training requirements, significantly improve the generalization ability of the model, reduce the cost of processing invalid data, avoid the occupation of computing resources by redundant data, shorten the model iteration cycle, and make the recognition performance of the model in complex scenes more stable.

[0006] To achieve the above purpose, the first aspect of the application provides a sample data set optimization processing method applied to image recognition, which comprises the following steps:

[0007] obtaining error sample data and model abnormal indicators of a target model; wherein the error sample data includes sample data whose prediction result is inconsistent with a sample label and error types;

[0008] The error sample data is analyzed by a clustering analysis method to determine the gap sample type; a target quality dimension corresponding to a model abnormality index is determined according to an association model; wherein the target quality dimension is an abnormal sample quality dimension;

[0009] The sample data set of the target model is optimized based on the gap sample type and the target quality dimension.

[0010] In an implementable manner, the error sample data is analyzed by a clustering analysis method, including:

[0011] Visual features of the error sample data are extracted; the visual features are analyzed by a clustering algorithm to obtain a plurality of clustering clusters;

[0012] The cluster attribute of the plurality of clustering clusters is analyzed and determined, and the cluster attribute is converted into a gap sample type; wherein the cluster attribute includes a target type, a scene type, a target state and a shooting condition.

[0013] In an implementable manner, a target quality dimension corresponding to a model abnormality index is determined according to an association model, including:

[0014] An association model corresponding to a model task of the target model is extracted; wherein the model task includes an image classification task, a target detection task and a semantic segmentation task;

[0015] A sample quality dimension corresponding to the model abnormality index is matched by the association model; the target quality dimension is determined by analyzing the sample data based on the matched sample quality dimension.

[0016] In an implementable manner, the construction of the association model includes:

[0017] A model index and a sample quality dimension of a model task are defined;

[0018] A historical data set of the model task is extracted, and the correlation between the model index and the sample quality dimension is evaluated based on the historical data set; the association model of the model index and the sample quality dimension is constructed according to the correlation.

[0019] In an implementable manner, the correlation between the model index and the sample quality dimension is evaluated based on the historical data set, including:

[0020] The sample dimension quality of the historical data set is calculated, and the model index when the model is trained by using the historical data set is extracted;

[0021] The correlation between the sample dimension quality and the model index is analyzed based on a correlation analysis method; wherein the correlation analysis method includes Pearson correlation analysis or Spearman correlation analysis.

[0022] In an implementable manner, the correlation model of the model index and the sample quality dimension is constructed according to the correlation, including:

[0023] A coefficient threshold of the correlation is set;

[0024] When the correlation coefficient of the sample quality dimension and the model index is greater than the coefficient threshold, the sample quality dimension and the model index are associated;

[0025] The correlation model is constructed according to a plurality of model indexes and associated sample quality dimensions.

[0026] In an implementable manner, the sample data is analyzed based on the matched sample quality dimension, including:

[0027] The matched sample quality dimension is sequentially marked as a basic quality dimension;

[0028] A sample data set of a training target model is extracted, and whether the sample data set has an abnormality is analyzed based on the basic quality dimension; if yes, the basic quality dimension is marked as a target quality dimension.

[0029] In an implementable manner, the sample data set of the target model is optimized based on the gap sample type and the target quality dimension, including:

[0030] The sample problem is determined according to the gap sample type and the target quality dimension; wherein the sample problem includes a sample quantity problem and a sample quality problem;

[0031] The sample data set is optimized according to the sample problem.

[0032] In an implementable manner, the sample data set is optimized according to the sample problem, including:

[0033] A sample data set of a target model is extracted;

[0034] The sample data in the sample data set is quality optimized according to the sample quality problem; the sample data set after the quality optimization is quantity optimized according to the sample quantity problem.

[0035] In an implementable manner, the sample data set after the quality optimization is quantity optimized according to the sample quantity problem, including:

[0036] A sample standard distribution of the sample data set is extracted; wherein the sample standard distribution includes a sample data type when all model indexes are normal and a sample proportion between each sample data type;

[0037] A sample supplement amount of each sample data type in the sample data set after the quality optimization is counted based on the sample quantity problem; the sample data set is supplement optimized based on the sample supplement amount.

[0038] Compared with the prior art, the present application has the beneficial effects that:

[0039] 1. The present application first acquires error sample data and model anomaly indicators of the target model; by extracting the visual features of the error sample data, the clustering algorithm is used to obtain the clustering cluster, and the gap sample type is determined by analyzing the cluster attribute; then the corresponding model of the model task is called, the sample quality dimension corresponding to the model anomaly indicator is matched, and the target quality dimension with abnormality is screened out through the pre-training model or the artificial verification sample data set; finally, the sample data set of the target model is optimized according to the logic of quality optimization first and quantity optimization second; the present application breaks through the limitation of traditional static data set processing that data requirements are judged by experience, on the one hand, the gap scene not covered by the model is accurately positioned through clustering, and the problems of sample redundancy or insufficient key scene samples are avoided; on the other hand, low-quality data is removed in combination with the target quality dimension, so that the supplemented samples have both quantity and quality, the optimized sample data set can accurately match the model training requirements, the generalization ability of the model is significantly improved, the cost of invalid data processing is reduced, the algorithm is avoided to be occupied by redundant data, the model iteration cycle is shortened, and the recognition performance of the model in complex scenes is more stable.

[0040] 2. When the associated model is constructed, the present application first defines the exclusive model indicators and sample quality dimensions for different model tasks such as image classification, target detection and semantic segmentation; then the multiple versions of historical data sets are collected, the sample quality dimensions of each version are calculated, the correlation between the sample quality dimensions and the corresponding model indicators is quantified through correlation analysis, the coefficient threshold is set to screen the strongly correlated sample quality dimensions, and the mapping rule of model indicator anomaly→sample quality dimension is constructed; the present application solves the pain points of traditional model training that only verifies the data output quality and does not judge the adaptability to the model, quantifies the correlation through the historical data set, changes the quality optimization from subjective judgment to data-driven, simultaneously adapts to the multi-task scene, can provide targeted quality optimization direction for different tasks, improves the positioning accuracy of data quality problems, reduces the artificial investigation cost, ensures that the core indicators of the model are stable at a high level, and improves the efficiency and reliability of the model training. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creating any inventive labor.

[0042] Figure 1 The method steps of the sample data set optimization processing method in the embodiments of the present application are shown in the following figure:

[0043] Figure 2A flowchart for constructing a correlation model in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The technical solutions of the present application will be described clearly and completely below in conjunction with embodiments. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts are within the scope of the present application.

[0045] Please refer to Figure 1 The first aspect embodiment of the present application provides a sample data set optimization processing method applied to image recognition, which comprises the following steps.

[0046] Obtaining error sample data and model abnormality indicators of a target model; wherein the error sample data comprises sample data with inconsistent prediction results and sample labels and error types;

[0047] Analyzing the error sample data by a clustering analysis method to determine a gap sample type; determining a target quality dimension corresponding to the model abnormality indicators according to a correlation model; wherein the target quality dimension is an abnormal sample quality dimension;

[0048] Optimizing the sample data set of the target model based on the gap sample type and the target quality dimension.

[0049] The target model refers to an artificial intelligence model trained by a sample data set to complete a specific model task. When the target model is trained by sample images in the sample data set, if the number of sample images is insufficient, the recognition of the target model to some scenes after training may not meet the set requirements, and if the number is redundant, the target model may need to process a large amount of redundant data during training, affecting the training efficiency. If the quality of the sample images is insufficient or not adaptive, the overall effect of the trained target model may not meet the set requirements.

[0050] In this embodiment, error sample data and model abnormality indicators during training of the target model are obtained first; it is determined according to the error sample data which sample data cannot be accurately recognized by the target model, so as to determine the gap sample type required by the target model; it is determined according to the model abnormality indicators which core indicators of the target model are abnormal, so as to determine which sample quality dimensions are abnormal according to the abnormal core indicators, and finally determine the required sample data. Finally, the gap sample type and the sample quality dimension are combined to optimize the sample data set of the target model, so as to ensure that the optimized sample data set meets the requirements in terms of quantity and quality.

[0051] The embodiment is to concentrate on solving the problems of the number and quality of sample data in the process of artificial intelligence model training, so that the error sample data and model abnormal indicators of a plurality of target models can be synchronously acquired, the sample data required to be optimized for each target model is analyzed, and then the sample data is centrally optimized and redistributed according to the summary result, which can improve the optimization efficiency and ensure that the number and quality of samples are solved.

[0052] In a preferred embodiment, the error sample data is analyzed by a clustering analysis method, including: extracting visual features of the error sample data; performing clustering analysis on the visual features by a clustering algorithm to obtain a plurality of clustering clusters; analyzing and determining cluster attributes of the plurality of clustering clusters, and converting the cluster attributes into a gap sample type; wherein the cluster attributes include a target type, a scene type, a target state, and a shooting condition.

[0053] During model training, an error sample capture module is embedded, an error sample hook is inserted in the model training framework, when the prediction result of the model is inconsistent with the sample label, an error sample recording mechanism is automatically triggered, the error sample is recorded in real time, and the error sample is marked with an error type, including classification error, positioning error, missed detection / mis-detection, etc.

[0054] The visual features of the captured error sample are extracted by a pre-trained feature extraction module, and the visual features are associated with the error sample data. The visual features of the error sample are analyzed by a clustering algorithm, and the gap sample type is located by analyzing the cluster attributes (such as target type, scene type, target state, shooting condition, etc.) of each cluster.

[0055] As an example but not limited to, taking the training of a city road vehicle target detection model (based on YOLOv8) as an example:

[0056] An error sample hook (Hook) is inserted in the validation stage (evaluation on the validation set after each training) of the YOLOv8 training framework, the hook function is bound after the model prediction output and before the loss calculation, the model prediction result is compared with the sample label in real time, and the error sample data is obtained.

[0057] For the sample data in the error sample data, a pre-trained ResNet-50 model is used to analyze the sample data, and a feature vector containing target appearance, scene lighting, background environment, etc. is output as a visual feature. The visual features of all sample data in the error sample data are analyzed by a K-Means clustering algorithm, and each cluster is manually labeled or automatically extracted cluster attributes.

[0058] For example, the cluster attribute of one of the clusters is: target type: small car; scene type: stormy night (street light + rain wiper reflection); target state: partially blocked by water reflection; shooting condition: low light (ISO>800), image blur (motion blur). In order to solve the problem of this cluster, the required sample data needs to include a small car in a stormy night with water reflection, and the gap sample type can be defined as: a small car in a stormy night with water reflection. According to the gap sample type, the sample data can be supplemented to solve the problem of this cluster.

[0059] The centralized optimization processing of the current sample data is a static process, and the sample data is fixed in the sample data set after the sample data processing is completed, which will cause the sample data set to be unable to adapt to the dynamic needs in model training. In the target model training, the error sample log is recorded, the error samples are classified by clustering algorithm, the gap sample type is located, and the sample data can be dynamically supplemented according to the gap sample type; after the supplemented sample data is processed, it is integrated into the original data set, and the new data set target model is used, which will not affect the iteration efficiency of the model, and can solve the problem that in the traditional centralized optimization processing, whether the sample data is sufficient is determined by experience, the data gap of the model cannot be quantified, and the scene sample is insufficient or redundant.

[0060] In a preferred embodiment, determining the target quality dimension corresponding to the model anomaly indicator according to the association model comprises: extracting the association model corresponding to the model task of the target model; wherein the model task includes an image classification task, a target detection task and a semantic segmentation task; matching the sample quality dimension corresponding to the model anomaly indicator through the association model; and analyzing the sample data based on the matched sample quality dimension to determine the target quality dimension.

[0061] In the model training process, the association model of the model indicator and the sample quality dimension is established, if a certain model indicator decreases, the corresponding dimension quality of the sample data is automatically traced back, and after it is determined that there is a problem with the corresponding dimension quality, it is optimized in a targeted manner to ensure that the model indicators are at a high level.

[0062] The technical scheme can solve the problem that in the traditional model training, only quality inspection is performed before data output, the data quality cannot be judged to adapt to the model, and the problem that the data annotation accuracy is high but the model training effect is still poor may occur.

[0063] In a preferred embodiment, please refer to Figure 2 The construction of the association model comprises: defining the model indicators and the sample quality dimensions of the model task; extracting the historical data set of the model task, and evaluating the correlation between the model indicators and the sample quality dimensions based on the historical data set; and constructing the association model of the model indicators and the sample quality dimensions according to the correlation.

[0064] To solve the problem of high data quality but poor model effect, we need to first define the model indicators for different tasks, then build a quantitative correlation between the model indicators and the sample quality dimensions, and finally locate the sample data problems through the correlation model to solve the sample problems.

[0065] Model indicators need to be defined in combination with specific tasks. Different tasks have different "poor" performances, and the quality problems of corresponding sample data are also different. The following lists some model indicators for different tasks:

[0066] 1. Core indicators for image classification tasks:

[0067] 1) Accuracy: The proportion of samples predicted correctly overall, suitable for balanced data sets;

[0068] 2) Precision: The proportion of sample data predicted as a certain class that is actually of that class, evaluating the misjudgment problem;

[0069] 3) Recall: The proportion of sample data that is actually of a certain class that is correctly predicted, evaluating the missed judgment problem;

[0070] 4) F1 Score: The harmonic mean of precision and recall, evaluating the class prediction effect;

[0071] 5) Confusion Matrix: Specific class misclassification, such as misclassifying a cat as a dog.

[0072] 2. Core indicators for object detection tasks:

[0073] 1) mAP: The average accuracy of all classes, evaluating the detection effect;

[0074] 2) Missed Detection Rate: The proportion of actual existing targets that are not detected;

[0075] 3) False Detection Rate: The proportion of actual non-existing targets among detected targets;

[0076] 4) Positioning Error: The degree of offset between predicted frame and true frame (such as IOU value).

[0077] 3. Core indicators for semantic segmentation tasks:

[0078] 1) MIoU: The mean intersection over union of all classes;

[0079] 2) Pixel Accuracy: The proportion of correctly classified pixels among all pixels;

[0080] 3) Class Accuracy: The proportion of correctly classified pixels of a certain class.

[0081] Similarly, different model tasks correspond to different sample quality dimensions. For example, for object detection tasks, the sample quality dimensions are defined as shown in Table 1:

[0082] Table 1. Sample quality dimensions of target detection task and model indicators of their influence

[0083] Sample quality dimensions Definition explanation Model indicators that may be affected Annotation accuracy Matching degree of true frame and labeled frame mAP, positioning error, false detection rate Class balance Distribution of sample quantity of each class Recall rate, mAP Scene diversity Coverage of sample data of the same class in different scenes (background, angle, light, etc.) mAP Target integrity Whether the target is occluded / truncated Missed detection rate Image clarity Resolution / fuzziness of target area Small target mAP

[0084] Next, taking the target detection task as an example, the correlation model between the model indicators and the sample quality dimensions is constructed:

[0085] First, collect the historical data sets of different versions under the target detection task. Calculate the sample quality dimensions for each historical data set, and extract the model indicators of the target detection model corresponding to each version of the historical data set. Then, the correlation coefficient between the sample quality dimensions and the model indicators is calculated through correlation analysis; the greater the absolute value of the correlation coefficient, the higher the correlation strength, and the positive correlation coefficient indicates positive correlation, and the negative correlation coefficient indicates negative correlation.

[0086] According to the correlation between the model indicators and the sample quality dimensions, an interpretable correlation model is constructed, that is, based on the above analysis, a mapping rule of model indicator abnormalities→possible sample quality dimensions is generated.

[0087] In a preferred embodiment, the correlation between the model indicators and the sample quality dimensions is evaluated based on the historical data set, including: calculating the sample dimension quality of the historical data set, and extracting the model indicators when training the model using the historical data set; based on the correlation analysis method, the correlation between the sample dimension quality and the model indicators is analyzed; wherein the correlation analysis method includes Pearson correlation analysis or Spearman correlation analysis.

[0088] For example, the sample quality dimensions of the four versions of the historical data set and the corresponding model indicators are as shown in Table 2:

[0089] Table 2. Sample quality dimensions of historical data sets and corresponding model indicators

[0090] Historical dataset version Sample quality dimensions Model indicators V1.0 Annotation accuracy = 0.80, class balance = 0.60, scene diversity = 1.0, target integrity = 0.65, image clarity = 6.0 mAP = 0.62, missed detection rate = 0.30, false detection rate = 0.25, positioning error = 0.35, small target mAP = 0.45 V2.0 Annotation accuracy = 0.92, class balance = 0.85, scene diversity = 1.1, target integrity = 0.68, image clarity = 6.2 mAP = 0.75, missed detection rate = 0.22, false detection rate = 0.12, positioning error = 0.20, small target mAP = 0.48 V3.0 Annotation accuracy = 0.93, class balance = 0.88, scene diversity = 1.5, target integrity = 0.82, image clarity = 7.8 mAP = 0.83, missed detection rate = 0.15, false detection rate = 0.08, positioning error = 0.18, small target mAP = 0.65 V4.0 Annotation accuracy = 0.92, class balance = 0.86, scene diversity = 1.1, target integrity = 0.80, image clarity = 6.5 mAP = 0.76, missed detection rate = 0.17, false detection rate = 0.09, positioning error = 0.19, small target mAP = 0.52

[0091] The correlation between the sample dimension quality and the model indicators calculated by the Pearson correlation coefficient is shown in Table 3:

[0092] Table 3. Correlation between sample dimension quality and model indicators

[0093] Model indicators Correlation coefficient with annotation accuracy Correlation coefficient with class balance Correlation coefficient with scene diversity Correlation coefficient with target integrity Correlation coefficient with image clarity mAP 0.91 (strong positive correlation) 0.87 (strong positive correlation) 0.89 (strong positive correlation) 0.72 (moderate positive correlation) 0.68 (moderate positive correlation) Missed detection rate -0.65 (moderate negative correlation) -0.85 (strong negative correlation) -0.70 (moderate negative correlation) -0.92 (strong negative correlation) -0.55 (weak negative correlation) False detection rate -0.93 (strong negative correlation) -0.60 (moderate negative correlation) -0.52 (weak negative correlation) -0.48 (weak negative correlation) -0.45 (weak negative correlation) Positioning error -0.90 (strong negative correlation) -0.50 (weak negative correlation) -0.55 (weak negative correlation) -0.42 (weak negative correlation) -0.58 (moderate negative correlation) Small target mAP 0.62 (moderate positive correlation) 0.58 (moderate positive correlation) 0.65 (moderate positive correlation) 0.52 (weak positive correlation) 0.90 (strong positive correlation)

[0094] It should be noted that the historical data set can be a sample data set corresponding to a previous version of the target model, or a sample data set used for training a model with the same type of model task.

[0095] In a preferred embodiment, the correlation model between the model indicators and the sample quality dimensions is constructed according to the correlation, comprising: setting a correlation coefficient threshold; when the correlation coefficient between the sample quality dimension and the model indicator is greater than the coefficient threshold, associating the sample quality dimension and the model indicator; and constructing the correlation model according to a plurality of model indicators and their associated sample quality dimensions.

[0096] As shown in Table 3, different correlation coefficient thresholds can be set for different model abnormal indicators, such as setting a medium correlation or above, or setting 0.55, and the correlation model established is shown in Table 4 as follows:

[0097] Table 4: Correlation model between model indicators and sample quality dimensions

[0098] Model anomaly indicators Possible sample quality dimensions Mapping basis Significant decrease in mAP ① Scene diversity decreases (e.g., missing key scene samples, poor model generalization); ② Annotation accuracy decreases (annotation errors cause model learning bias); ③ Class balance decreases (a sudden decrease in samples of a certain class, which lowers overall precision) Scene diversity and mAP are strongly positively correlated (0.89); annotation accuracy and mAP are strongly positively correlated (0.91); class balance and mAP are strongly positively correlated (0.87) Missed detection rate suddenly increases ① Target integrity decreases (a large number of new occlusion / truncated samples, making it difficult for the model to identify); ② Class balance decreases (more missed detection of classes with fewer samples) Target integrity and missed detection rate are strongly negatively correlated (-0.92); class balance and missed detection rate are strongly negatively correlated (-0.85) False detection rate increases significantly ① Annotation accuracy decreases (annotation box errors or false positive annotations, model learns incorrect target features) Annotation accuracy and false detection rate are strongly negatively correlated (-0.93) (the only strongly correlated dimension) Positioning error increases ① Annotation accuracy decreases (annotation box offset, model learns incorrect positioning pattern); ② Image clarity decreases (fuzzy target causes prediction box to shift) Annotation accuracy and positioning error are strongly negatively correlated (-0.90); image clarity and positioning error are moderately negatively correlated (-0.58) Small target mAP continues to be low ① Image clarity decreases (small targets have fewer pixels, and blurring increases detection difficulty); ② Annotation accuracy is insufficient (small target annotation boxes are easily offset, affecting learning) Image clarity and small target mAP are strongly positively correlated (0.90); annotation accuracy and small target mAP are moderately positively correlated (0.62)

[0099] In a preferred embodiment, the sample data is analyzed based on the matched sample quality dimensions, comprising: sequentially marking the matched sample quality dimensions as basic quality dimensions; extracting a sample data set of the training target model, and analyzing whether the sample data set has an abnormality based on the basic quality dimensions; and if so, marking the basic quality dimensions as target quality dimensions.

[0100] After matching the sample quality dimensions associated with the model abnormal indicators through the correlation model corresponding to the target model, it does not mean that the sample data set has problems corresponding to all the matched sample quality dimensions. It is possible that only a certain sample quality dimension in the sample data set has a problem.

[0101] The matched sample quality dimensions are sequentially marked as basic quality dimensions, and whether the sample data set of the target model has a problem corresponding to the basic quality dimension is analyzed through a pre-trained model or manually, and if so, the basic quality dimension is marked as a target quality dimension.

[0102] For example, if the mAP drops significantly, the sample quality dimensions matched according to the correlation model include scene diversity decline, annotation accuracy decline, and class balance decline.

[0103] For scene diversity, the sample data in the sample data set is automatically annotated, and the proportion of each scene is counted. If the proportion of a certain scene changes abnormally relative to the standard data set, it is determined that the sample data set has a scene diversity problem.

[0104] For annotation accuracy, sample data in the sample data set is randomly extracted, and the annotation labels of the extracted sample data are manually reviewed. If the annotation accuracy of the review is consistent with the standard data set, the sample data set does not have an annotation accuracy problem.

[0105] For category balance, the ratio of the least number of samples in the sample dataset to the most number of samples is calculated. If the ratio is consistent with the standard dataset, the sample dataset does not have a category balance problem.

[0106] It is worth noting that the standard dataset can be a version of historical dataset. For example, in the pedestrian and non-motor vehicle target detection task of the intelligent traffic camera, the "mAP significantly decreased, specifically from 0.83 in V3.0 to 0.72 in V4.0, a decrease of 13%, and the historical dataset of V3.0 version can be used as the standard dataset. Of course, the standard dataset can also be a pre-adjusted dataset that meets various requirements for model training.

[0107] In a preferred embodiment, the sample dataset of the target model is optimized based on the gap sample type and the target quality dimension, comprising: determining sample problems according to the gap sample type and the target quality dimension; wherein the sample problems include sample quantity problems and sample quality problems; and optimizing the sample dataset according to the sample problems.

[0108] In a preferred embodiment, the sample dataset is optimized according to the sample problems, comprising: extracting the sample dataset of the target model; optimizing the sample data in the sample dataset according to the sample quality problems; and optimizing the sample dataset after quality optimization according to the sample quantity problems.

[0109] After determining the gap sample type and the target quality dimension of the sample dataset, the sample problems existing in the sample dataset need to be determined comprehensively. If there is a problem with the target quality dimension of the sample dataset, the sample data in the sample dataset is optimized according to the target quality dimension; for example, if the sample data is not accurately labeled, the sample data is relabeled, and if the image clarity of the sample data is insufficient, the sample data is removed. It can be seen that in the quality optimization process, it may only be necessary to optimize the sample data, or it may be necessary to supplement the sample data.

[0110] After quality optimization, the quantity problem of the sample data in the sample dataset needs to be solved. As mentioned above, after quality optimization, not only the gap sample type needs to be supplemented, but also the sample data removed after quality optimization needs to be supplemented to make up for the decrease in sample data caused by quality optimization.

[0111] In a preferred embodiment, the sample dataset after quality optimization is optimized according to the sample quantity problems, comprising: extracting the sample standard distribution of the sample dataset; wherein the sample standard distribution includes the sample data types when all model indicators are normal and the sample proportion between each sample data type; based on the sample quantity problems, the sample supplement amount of each sample data type in the sample dataset after quality optimization is counted; and the sample dataset is supplemented and optimized based on the sample supplement amount.

[0112] The quantity optimization of the sample data set is essentially supplementing in the sample data set to ensure that the sample data in the sample data set is sufficient and balanced. After the quality optimization, the image attributes of the statistical excluded sample data are consistent with the cluster attributes, and the supplement amount of the sample data is determined by using the statistical results. In this embodiment, the sample standard distribution of the sample data set is taken as the reference. If the sample data set needs to supplement a certain type of sample data, the amount of sample data to be supplemented is determined according to the existing amount of the sample data of this type and the proportion between the sample data of this type and other types. It is worth noting that the sample standard distribution can be obtained by statistical standard data set.

[0113] For example, it is assumed that the sample quantity problem determined according to the gap sample type is that image data of a small car with a stormy night and water reflection needs to be supplemented. During the quality optimization, 50 pieces of image data of a small car with a stormy night and water reflection are excluded, and there are 950 pieces of image data of a small car with a stormy night and water reflection in the sample data set. According to the proportion, there should be 1500 pieces, and therefore 550 pieces of image data of a small car with a stormy night and water reflection need to be supplemented.

[0114] The above embodiments are only used to illustrate the technical method of the present application but not limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical method of the present application.

Claims

1. A method for centralized optimization processing of sample data applied to image recognition, characterized in that, include: Obtain error sample data and model anomaly indicators for the target model; where error sample data includes sample data where the prediction results are inconsistent with the sample labels and error types; The erroneous sample data is analyzed using clustering analysis to determine the gap sample type; the target quality dimension corresponding to the model anomaly index is determined based on the association model; wherein, the target quality dimension is the quality dimension of the anomalous samples. The sample dataset for optimizing the target model is based on the gap sample type and the target quality dimension.

2. The sample data central optimization processing method for image recognition according to claim 1, characterized in that, The erroneous sample data was analyzed using cluster analysis methods, including: Visual features of the erroneous sample data are extracted; clustering analysis is performed on the visual features using a clustering algorithm to obtain several clusters; The cluster attributes of several clusters are analyzed and determined, and the cluster attributes are converted into gap sample types; wherein, the cluster attributes include target type, scene type, target state and shooting conditions.

3. The sample data central optimization processing method for image recognition according to claim 1, characterized in that, The target quality dimension corresponding to the abnormal indicators of the model is determined based on the correlation model, including: Extract the associated models of the model tasks corresponding to the target model; wherein, the model tasks include image classification tasks, object detection tasks, and semantic segmentation tasks; The sample quality dimension corresponding to the abnormal indicator of the model is matched by the association model; the sample data is analyzed based on the matched sample quality dimension to determine the target quality dimension.

4. The sample data central optimization processing method for image recognition according to claim 3, characterized in that, The construction of the association model includes: Define the model metrics and sample quality dimensions for the model task; Extract the historical dataset of the model task, evaluate the correlation between model metrics and sample quality dimensions based on the historical dataset, and construct a correlation model between model metrics and sample quality dimensions based on the correlation.

5. The sample data central optimization processing method for image recognition according to claim 4, characterized in that, The correlation between model metrics and sample quality dimensions is evaluated based on the historical dataset, including: Calculate the sample dimension quality of the historical dataset and extract the model metrics when training the model using the historical dataset; The correlation between the sample dimensional quality and the model indicators is analyzed based on correlation analysis methods; wherein, the correlation analysis methods include Pearson correlation analysis or Spearman correlation analysis.

6. The sample data central optimization processing method for image recognition according to claim 5, characterized in that, Based on the correlation, a correlation model is constructed between model metrics and sample quality dimensions, including: Set a threshold for the correlation coefficient; When the correlation coefficient between the sample quality dimension and the model indicator is greater than the coefficient threshold, the sample quality dimension and the model indicator are associated. A correlation model is constructed based on several model metrics and their associated sample quality dimensions.

7. The sample data central optimization processing method for image recognition according to claim 3, characterized in that, Analyzing sample data based on the matched sample quality dimensions includes: The quality dimensions of the matched samples are sequentially labeled as basic quality dimensions; Extract the sample dataset used to train the target model, and analyze whether there are any anomalies in the sample dataset based on the basic quality dimension; if so, mark the basic quality dimension as the target quality dimension.

8. The sample data central optimization processing method for image recognition according to any one of claims 1 to 7, characterized in that, The sample dataset for optimizing the target model based on the gap sample type and the target quality dimension includes: The sample problem is determined based on the gap sample type and the target quality dimension; wherein, the sample problem includes the sample quantity problem and the sample quality problem; Optimize the sample dataset based on the sample problem.

9. The sample data central optimization processing method for image recognition according to claim 8, characterized in that, Optimize the sample dataset based on the sample problem, including: Extract the sample dataset of the target model; The quality of the sample data in the sample dataset is optimized according to the sample quality problem; the quantity of the optimized sample dataset is optimized according to the sample quantity problem.

10. The sample data central optimization processing method for image recognition according to claim 9, characterized in that, The quantity optimization of the quality-optimized sample dataset is performed according to the aforementioned sample quantity problem, including: Extract the standard distribution of the sample dataset; wherein, the standard distribution of the sample dataset includes the data types of the samples when all model indicators are normal and the sample ratios between each data type; Based on the issue of sample quantity, the amount of supplementary samples for each data type in the sample dataset after quality optimization is statistically analyzed; and the sample dataset is further optimized based on the amount of supplementary samples.

Images