Method for diagnosing advantages and disadvantages of visual algorithm training set

Abstract

The invention discloses a method for diagnosing the advantages and disadvantages of a visual algorithm training set, and the method comprises the following steps: S1, data marking: carrying out the classification and marking of a visual data set for training, so as to facilitate the subsequent statistical analysis; s2, multi-dimensional statistical analysis: performing analysis and statistics on the visual data set from the dimensions of scene location, indoor and outdoor, picture view angle, shielding, pixel size and label number; and S3, training set generalization diagnosis: calculating a generalization index of the visual data set through a generalization index formula, if the generalization index is higher than the benchmarking value, the generalization of the visual data set is good, and if the generalization index is lower than the benchmarking value, the generalization of the visual data set is insufficient. And the visual data set with insufficient generalization can be enhanced and optimized or deleted, so that negative effects on the model are prevented. The scheme is suitable for the training process of the vision algorithm.

Description

Technical Field

[0001] The present invention relates to the fields of machine learning and artificial intelligence, and in particular to a method for diagnosing the quality of a visual algorithm training set. Background Art

[0002] In the field of machine learning and artificial intelligence, the generalization ability of visual algorithms is one of the key indicators to measure their performance. The generalization of algorithms usually depends on the diversity and quality of training sets. The quality of training sets directly affects the performance of algorithms in practical applications. Therefore, developing an effective method to diagnose and optimize visual algorithm training sets is of great practical significance.

[0003] Currently, the construction and evaluation of visual algorithm training sets mainly rely on manual experience and lack an automated and standardized evaluation process. In addition, for the generalization analysis of training sets, most studies focus on a single dimension, such as scene type or image quality, without considering multiple factors comprehensively.

[0004] Existing methods for evaluating visual algorithm training sets have the following shortcomings:

[0005] 1. Lack of systematicity: Existing methods often focus on a single metric, such as image resolution or label accuracy, while ignoring the multi-dimensional characteristics of the training set.

[0006] 2. Insufficient generalization evaluation: Existing technologies make it difficult to comprehensively evaluate the generalization ability of training sets in different scenarios.

[0007] 3. Lack of screening and optimization mechanisms: Existing methods fail to provide effective strategies for screening and optimizing training sets to improve the generalization of the algorithm. Summary of the invention

[0008] The present invention mainly solves the technical problem that the prior art cannot systematically determine the generalization of a training set, and provides a method for diagnosing the quality of a visual algorithm training set, which can accurately consider the generalization of a training set in multiple dimensions.

[0009] The present invention aims to solve the above technical problems mainly through the following technical solutions: a method for diagnosing the quality of a visual algorithm training set, comprising the following steps:

[0010] S1. Data labeling: Classify and label the visual data set used for training to facilitate subsequent statistical analysis;

[0011] S2. Multi-dimensional statistical analysis: Analyze and count the visual data set from the dimensions of scene location, indoor and outdoor, picture angle, occlusion, pixel size and number of labels;

[0012] S3. Generalization diagnosis of training set: The generalization index of the visual dataset is calculated by the generalization index formula. If the generalization index is higher than the benchmark value, the visual dataset has good generalization. If the generalization index is lower than the benchmark value, the visual dataset has insufficient generalization.

[0013] The generalization index system established in this solution includes scene location diversity index, indoor and outdoor balance index, perspective diversity index, occlusion complexity index, pixel size adaptability index and label quantity balance index. Visual datasets with insufficient generalization can be enhanced or deleted to prevent negative impact on the model. The benchmark value is an empirical parameter determined by experiments.

[0014] Preferably, the generalization index formula is:

[0015]

[0016] In the formula, G avg is the overall score of the generalization index; α is the weight coefficient of the scene and indoor and outdoor dimensions, β is the weight coefficient of the picture perspective and occlusion dimension, γ is the weight coefficient of the pixel size and label number dimension, which is used to adjust the influence of each group of factors on the overall generalization, and N is the total number of pictures; D loc is the scene location diversity index, D inout is the indoor and outdoor balance indicator, V is the perspective diversity indicator, O is the occlusion complexity indicator, S is the pixel size adaptability indicator, and L is the label quantity balance indicator.

[0017] As a preferred option, the scene location diversity index D loc Determined by the following formula:

[0018]

[0019] In the formula, p i is the proportion of the i-th place type samples in the training set, and n is the total number of place types.

[0020] As a preferred choice, the indoor and outdoor balance index D inout It is the ratio of the number of indoor samples to the number of outdoor samples.

[0021] Preferably, the perspective diversity index V is determined by the following formula:

[0022]

[0023] In the formula, v iis the viewing angle of the ith sample, and μv is the average viewing angle of all samples. The viewing angle includes the shooting angle of mobile phones, the oblique viewing angle of monitoring, the vertical viewing angle of drones, the ground viewing angle of sweeping robots, the near-ground viewing angle of robot dogs, and the natural viewing angle of simulated robots. Specifically, it refers to the category angle classified based on the angle between the central axis of the lens and the horizontal direction during shooting. The categories and corresponding category angles include the following four: horizontal viewing angle: 0 degrees, overlooking viewing angle: 45 degrees, vertical viewing angle: 90 degrees, and upward viewing angle: 135 degrees. For example, during a certain shot, the angle between the central axis of the lens and the horizontal direction is 30 degrees, and the closest category is the 45-degree overlooking viewing angle, so it is classified as the overlooking viewing angle, that is, v i is 45.

[0024] Preferably, the occlusion complexity index O is the ratio of the total number of occluded pixels to the total number of image pixels.

[0025] Preferably, the pixel size adaptability index S is determined by the following formula:

[0026]

[0027] In the formula, s i is the resolution of the i-th sample, and δ is the mapping function that maps the image resolution to between 0 and 1. That is, the image resolution with the largest resolution in the training set is P max , δ is the resolution of each image and P max ratio.

[0028] Preferably, the label quantity balance index L is determined by the following formula:

[0029]

[0030] In the formula, q j is the proportion of the j-th category label in the training set, and m is the total number of category labels.

[0031] Preferably, the method for diagnosing the quality of a visual algorithm training set also includes a training set generalization enhancement optimization method, specifically: using a cutout synthesis method to supplement foreground and background images to improve different indicator values.

[0032] For example, to improve the scene diversity index, you only need to cut out the existing foreground target and synthesize it into the newly added background image to quickly improve the scene diversity index.

[0033] The substantial effects brought by the present invention are: 1. Improving the generalization ability of the algorithm: Through the method of the present invention, the training set can be prepared and optimized more effectively, thereby improving the generalization ability of the visual algorithm;

[0034] 2. Optimize resource allocation: Screening and optimizing training sets can reduce reliance on low-quality data and optimize the allocation of computing resources.

[0035] 3. Improve algorithm performance: The algorithm trained using the training set screened and optimized using the method of the present invention performs more stably and accurately on unknown data.

[0036] 4. Promote algorithm development: The method of the present invention provides a new tool for the development of visual algorithms, which helps to accelerate the development of new algorithms and the improvement of existing algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flow chart of a method for diagnosing the quality of a visual algorithm training set according to the present invention. DETAILED DESCRIPTION

[0038] The technical solution of the present invention is further specifically described below through embodiments and in conjunction with the accompanying drawings.

[0039] Embodiment: A method for diagnosing the quality of a visual algorithm training set in this embodiment, such as Figure 1 As shown, the following steps are included:

[0040] S1. Data labeling: First, collect the visual data set for training, and perform preliminary classification and labeling on it to facilitate subsequent statistical analysis.

[0041] S2. Multi-dimensional statistical analysis: The multi-dimensional statistical analysis method of the present invention is used to conduct an in-depth analysis of the training set. The analysis dimensions include scene location, indoor and outdoor, picture angle, occlusion, pixel size, number of labels, etc., to comprehensively evaluate the generalization of the training set.

[0042] S3. Generalization diagnosis of training set: The generalization index of the visual dataset is calculated by the generalization index formula. If the generalization index is higher than the benchmark value, the generalization of the visual dataset is good. If the generalization index is lower than the benchmark value, the generalization of the visual dataset is insufficient.

[0043] S4. Optimization of training set generalization enhancement: Use the cutout synthesis method to supplement the foreground and background images to improve different index values. For example, to improve the scene diversity index, you only need to cut out the existing foreground target and synthesize it into the newly added background image to quickly improve the scene diversity index.

[0044] This solution establishes a generalization index system based on the results of multi-dimensional statistical analysis. The system includes scene location diversity index, indoor and outdoor balance index, perspective diversity index, occlusion complexity index, pixel size adaptability index and label quantity balance index.

[0045] Description of multi-dimensional statistical indicators:

[0046] ① Scene location:

[0047] Statistical Unit: Category

[0048] Description: The scene locations can be divided into categories such as cities, villages, offices, industrial areas, etc. according to different application requirements.

[0049] ②Indoor and outdoor:

[0050] Statistical unit: binary classification

[0051] Description: Indoor and outdoor are usually used to distinguish the shooting environment of an image.

[0052] ③Picture angle:

[0053] Statistical Unit: Perspective Category

[0054] Description: Classify images according to different viewing angles, such as bottom view, horizontal view, top view, and vertical view.

[0055] ④Occlusion:

[0056] Statistical unit: Percentage

[0057] Description: The degree of occlusion is measured by calculating the percentage of pixels in the occluded area to the total pixels in the image.

[0058] ⑤Pixel size:

[0059] Statistical unit: Horizontal × Vertical Resolution

[0060] Description: The size of an image is usually described using the total number of pixels in the image or a specific horizontal and vertical resolution, such as 1920×1080 pixels.

[0061] ⑥Number of tags:

[0062] Statistical units: number of categories, label counts

[0063] Description: Count the number of labels of different categories in the training set, as well as the count of each category label, to evaluate the distribution balance of the labels.

[0064] The generalization index formula combines the influencing factors such as scene location, indoor and outdoor, picture angle, occlusion, pixel size and number of labels. The richer the dimension, the better the generalization:

[0065]

[0066] in:

[0067] G avgis the overall score of the generalization index. α, β, and γ are weight coefficients of different groups, which are used to adjust the influence of each group of factors on the overall generalization. N is the total number of images.

[0068] Scenes and indoor and outdoor dimensions:

[0069] D loc It is a scene location diversity indicator that measures the distribution of samples of different location types.

[0070] D inout It is an indoor-outdoor balance index, which measures the balance between indoor and outdoor samples.

[0071] Picture perspective and occlusion dimension:

[0072] V is the perspective diversity index, which measures the range of changes in the picture perspective.

[0073] O is the occlusion complexity index, which measures the degree of occlusion in the image and its impact on the vision algorithm.

[0074] Pixel size and number of labels dimensions:

[0075] S is the pixel size adaptability index, which measures the algorithm's adaptability to images of different resolutions.

[0076] L is an indicator of the balance of the number of labels, which measures the distribution balance of labels of different categories.

[0077] The indicators for each dimension can be defined in the following way:

[0078] D loc The diversity of place types can be quantified by calculating the Shannon entropy of the number of samples in each category:

[0079]

[0080] Among them, p i is the proportion of the i-th location type samples in the training set.

[0081] D inout It can be measured by the ratio of the number of indoor samples to the number of outdoor samples:

[0082]

[0083] V can be measured by the distribution of viewing angles, including mobile phone photography, surveillance viewing angles, and drone viewing angles, using the variance or standard deviation of the viewing angle:

[0084]

[0085] Where vi is the viewing angle of the ith sample, and μV is the average viewing angle of all samples.

[0086] O can be measured by calculating the proportion of the occluded area in the image:

[0087]

[0088] S can be measured by counting the number of images with different resolutions:

[0089]

[0090] Among them, s i is the resolution of the i-th sample, and δ is a mapping function that maps the resolution si of each image to a value between 0 and 1, which is used to measure the quality of the resolution.

[0091] L can be measured by calculating the balance of the number of labels in each category, similar to the location diversity index:

[0092]

[0093] Among them, q j is the proportion of the j-th category label in the training set

[0094] The actual calculation method may need to be adjusted according to the specific application scenario and data set characteristics. The weight coefficients α, β, and γ need to be determined through experiments or professional knowledge to optimize the accuracy and practicality of the generalization indicator.

[0095] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

[0096] Although this article uses more terms such as dimension, weight coefficient, benchmark value, etc., it does not exclude the possibility of using other terms. These terms are used only to more conveniently describe and explain the essence of the present invention; interpreting them as any additional restrictions is contrary to the spirit of the present invention.

Claims

1. A method for diagnosing the quality of a visual algorithm training set, characterized in that: The following steps are involved: S1. Data labeling: classify and label the visual dataset used for training; S2. Multi-dimensional statistical analysis: Analyze and count the visual data set from the dimensions of scene location, indoor and outdoor, picture angle, occlusion, pixel size and number of labels; S3. Generalization diagnosis of training set: The generalization index of the visual dataset is calculated by the generalization index formula. If the generalization index is higher than the benchmark value, the visual dataset has good generalization. If the generalization index is lower than the benchmark value, the visual dataset has insufficient generalization.

2. A method for diagnosing the quality of a visual algorithm training set according to claim 1, characterized in that: The generalization index formula is: In the formula, G avg is the overall score of the generalization index; α is the weight coefficient of the scene and indoor and outdoor dimensions, β is the weight coefficient of the picture perspective and occlusion dimension, γ is the weight coefficient of the pixel size and the number of labels, and N is the total number of pictures; D loc is the scene location diversity indicator, D inout is the indoor and outdoor balance indicator, V is the perspective diversity indicator, O is the occlusion complexity indicator, S is the pixel size adaptability indicator, and L is the label quantity balance indicator.

3. A method for diagnosing the quality of a visual algorithm training set according to claim 1 or 2, characterized in that: Scene location diversity index D loc Determined by the following formula: In the formula, p i is the proportion of the i-th place type samples in the training set, and n is the total number of place types.

4. A method for diagnosing the quality of a visual algorithm training set according to claim 1 or 2, characterized in that: Indoor and outdoor balance index D inout It is the ratio of the number of indoor samples to the number of outdoor samples.

5. The method for diagnosing the quality of a visual algorithm training set according to claim 3, characterized in that: The perspective diversity index V is determined by the following formula: In the formula, v i is the viewing angle of the ith sample, and μv is the average viewing angle of all samples.

6. A method for diagnosing the quality of a visual algorithm training set according to claim 3, characterized in that: The occlusion complexity index O is the ratio of the total number of occluded pixels to the total number of image pixels.

7. A method for diagnosing the quality of a visual algorithm training set according to claim 3, characterized in that: The pixel size adaptability index S is determined by the following formula: In the formula, s i is the resolution of the i-th sample, and δ is the mapping function that maps the image resolution to between 0 and 1.

8. The method for diagnosing the quality of a visual algorithm training set according to claim 3, characterized in that: The label quantity balance index L is determined by the following formula: In the formula, q j is the proportion of the j-th category label in the training set, and m is the total number of category labels.

9. A method for diagnosing the quality of a visual algorithm training set according to claim 3, characterized in that: It also includes optimization methods for enhancing the generalization of training sets, specifically: using cutout synthesis methods to supplement foreground and background images to improve different indicator values.

Images