A method to improve the sample quality of target detection tasks

Abstract

The present invention relates to a method for improving the sample quality of target detection tasks, including obtaining segmentation annotations corresponding to an image; calculating the pixel area of ​​a defect object in the annotation; calculating the long side h and the short side w of the minimum circumscribed rectangular box of the defect object; adjusting the size of the annotation box according to the proportion of the defect area relative to the minimum circumscribed rectangular box; obtaining an original image and a corresponding mask image; traversing the original image to determine the image cutting position in a sliding window manner; processing the original image and the mask image according to the image cutting position, if the size of the cut-out small image is smaller than the model size, the annotation box is expanded outward around the small image to the model adaptation size and then input into the model. The present invention uses semantic segmentation annotations instead of target detection box annotations, calculates the minimum circumscribed rectangular box of the segmentation annotations as the annotation detection box; dynamically adjusts the target detection annotation box for two scenarios: small pixel area and excessive proportion in the annotation box, ensures a reasonable proportion of foreground and background information, and thus obtains better annotation results; optimizes the detection strategy of small defects; uses image cutting and padding strategies to process large-size images, avoids changes in defect appearance caused by resizing, and is conducive to improving the detection effect of target detection.

Description

Technical Field

[0001] The present invention relates to the technical field of image vision detection, and in particular to a method for improving the quality of samples in target detection tasks. Background Art

[0002] Currently, in industrial production, defect detection on the surface and inside of products is an important part of ensuring product quality. Although the commonly used YOLO target detection model performs well in many applications, it has the problem of missing small defects, especially in the complex background and variable lighting conditions of industrial sites.

[0003] Existing improvement methods mainly include two aspects: data processing and model architecture improvement.

[0004] Data processing can enhance the original image data and generate more diverse training data through methods such as rotation, scaling, translation, and color transformation, so that the model can better adapt to various manifestations of small defects; during the training process, focus on learning those small defects that are difficult to detect and improve the detection capability of the model.

[0005] Improvements to the model architecture can include multi-scale feature fusion. A feature pyramid network (FPN) can be introduced into the YOLO model to enhance the model's ability to detect defects of different sizes by fusing features of different scales. A weighted loss function can be designed to make the model pay more attention to the detection of small defects during training, thereby improving detection accuracy. Alternatively, an attention mechanism can be introduced to enable the model to automatically focus on areas where small defects are located, thereby improving detection accuracy.

[0006] However, the size of the target detection box may not be completely reasonable when manually annotating industrial data sets. If the defect is too small and the detection box is also small, the loss of the small defect is more likely to be ignored when calculating the loss; if the defect accounts for too large a proportion of the detection box, the proportion of background information and foreground information is unbalanced, which is not conducive to the model comprehensively considering the characteristics of the defect and the location characteristics of the background.

[0007] In the detection of slender and tiny defects, the features of these defects are easily lost in the downsampling process. In particular, the area ratios of different defects in the annotation box vary significantly, especially for horizontal or vertical linear defects. This inconsistency in area ratio leads to information imbalance between the foreground and background. Specifically, the amount of information about the foreground defect is not proportional to the amount of background information. This imbalance seriously affects the detection accuracy and generalization ability of the model.

[0008] In addition, the goal of image preprocessing is to optimize the image data so that the model can learn and identify defect features more effectively. The expected image preprocessing result should be: while maintaining the integrity and recognizability of defect features, the image is adjusted to a uniform size required for model training. Ideal preprocessing should ensure the following:

[0009] Preservation of defect features: When resizing the image, the loss of feature information should be minimized, especially for small and linear defects.

[0010] Size consistency: Ensure that all input images are of uniform size so that the model can learn stably.

[0011] Contrast between defects and background: Through appropriate image enhancement technology, the contrast between defects and background can be improved, thereby alleviating the problem of imbalance between foreground and background information.

[0012] Enhancement of defect features: For small defects that are easily overlooked, their features are highlighted through image enhancement technology so that they can receive full attention from the model during the training process.

[0013] However, current image preprocessing methods may cause downsampling loss of defect features during the reduction operation, and may distort the actual size of the defect during the enlargement operation, both of which fail to achieve the above-mentioned expected preprocessing effects. Summary of the invention

[0014] The technical problem to be solved by the present invention is to provide a method for improving the quality of samples in target detection tasks, thereby solving the problem that the size of the target detection box may not be completely reasonable when manually annotating industrial data sets.

[0015] The technical solution adopted by the present invention to solve the technical problem is: a method for improving the quality of target detection task samples, comprising the following steps:

[0016] S1, obtain the segmentation annotation corresponding to the image;

[0017] S2, calculating the pixel area of ​​the defective object in the annotation;

[0018] S3, calculating the long side h and short side w of the minimum circumscribed rectangular box of the defect object;

[0019] S4. Adjust the size of the annotation box according to the proportion of the defect area to the minimum circumscribed rectangular box;

[0020] S5, obtaining the original image and the corresponding mask image;

[0021] S6, traversing the original image to determine the image cutting position in a sliding window manner;

[0022] S7. Process the original image and the mask image according to the cut image position. If the cut image is smaller than the model size, expand the annotation frame around the small image to the model adaptation size and then input the model.

[0023] Furthermore, in step S4 of the present invention, if the proportion of the defect area relative to the minimum circumscribed rectangular frame exceeds a preset threshold, and the short side w of the rectangular frame is smaller than a preset pixel threshold, the annotation frame is adjusted accordingly.

[0024] Furthermore, in step S4 described in the present invention, when the aspect ratio w:h is less than a preset ratio, only the width w of the annotation box is adjusted; the adjustment method is: using the long side h, a preset reasonable threshold and the authority area to calculate the amount by which the short side w needs to be increased.

[0025] Furthermore, the specific adjustment process of the adjustment method described in the present invention is to add areas of the same size on both sides of the short side to expand the width of the marking box.

[0026] Furthermore, in step S4 described in the present invention, when the aspect ratio w:h is not lower than the preset ratio, but the long side h is smaller than the set threshold, h and w are updated synchronously; the specific adjustment method is to update the size of the marking box to three times the original size, that is, the new long side is 3h, and the new short side is 3w.

[0027] Furthermore, in step S4 of the present invention, when the aspect ratio w:h is not less than a preset ratio and the long side h is not less than a set threshold, 2 pixels are added to the original short side w and long side h to expand the size of the annotation box.

[0028] Furthermore, in step S7 of the present invention, both the long side and the short side of the small image are expanded outward by padding.

[0029] The beneficial effects of the present invention are that the defects existing in the background technology are solved, semantic segmentation annotation is used instead of target detection frame annotation, and the minimum circumscribed rectangular frame of the segmentation annotation is calculated as the annotation detection frame; the target detection annotation frame is dynamically adjusted for two scenarios: small pixel area and excessive proportion in the annotation frame, to ensure a reasonable proportion of foreground and background information, thereby obtaining better annotation results; the detection strategy for small defects is optimized; and the cutting and padding strategies are used to process large-size images to avoid changes in defect appearance caused by resizing, which is beneficial to improving the detection effect of target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the label mask processing flow of the present invention;

[0031] Figure 2 It is a schematic diagram of the cutting process of the present invention. DETAILED DESCRIPTION

[0032] The present invention will now be described in further detail in conjunction with the accompanying drawings and preferred embodiments. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0033] like Figure 1-Figure 2 The core of the method shown in Figure 1 is to dynamically adjust the size of the target detection box according to the ratio of the defect segmentation annotation (mask) area to the minimum circumscribed rectangular box area. This strategy aims to achieve a balance between the foreground (i.e., defect area) and background information in the target detection box of the input model, thereby optimizing the detection performance of the model.

[0034] For image annotation, we do not directly use YOLO's target detection box annotation format, but use the segmentation annotation results to provide information for the subsequent dynamic generation of annotation boxes. After obtaining the segmentation annotation results, we determine the area ratio of the defect mask within its minimum bounding rectangle. If the ratio is higher than the threshold, it means that the foreground ratio in the corresponding annotation box is too large and the background ratio is too small. We adjust the internal foreground and background ratios by expanding the rectangular box. Otherwise, we use the original minimum bounding rectangle as the annotation box. The dynamically generated annotation box takes into account the ratio of small defects in the annotation box, making the foreground information and background information more balanced.

[0035] During data preprocessing, if the input image size is smaller than the input size of the model, the original approach is to scale the image to a specified size along the long side and fill the short side. To avoid the impact of scaling on defects, the present invention only uses padding to change the image size; if the input image size is smaller than the input size of the model, the large image is cut into small images by slicing to meet the model input requirements. The above operation retains more appearance information of the defect.

[0036] The specific steps are as follows:

[0037] 1. Optimize target detection annotation box

[0038] In order to improve the accuracy of the target detection annotation box, the present invention optimizes the annotation process of the YOLO target detection data set. In traditional annotation practice, annotators usually fit the rectangular box tightly to the target object to define the area that should be focused on during model training. However, this method has the following problems: if the target defect size is too small, it is easy to be ignored by the model when calculating the loss function; conversely, if the defect occupies too large a proportion in the annotation box, it will cause an imbalance between the foreground and background information, which is not conducive to the model capturing the individual characteristics of the defect and its positional relationship in the background at the same time.

[0039] To address these issues, the present invention proposes a set of dynamic adjustment strategies to solve the following two specific scenarios: one is for defects with small pixel areas, and the other is for defects that occupy too large a proportion of the labeled box. The design purpose of this solution is to reduce the deviation caused by manual annotation, thereby optimizing the data quality of model training and improving detection performance.

[0040] The traditional object detection box annotation method cannot fully capture the specific form of the defect because it lacks the key information required for dynamic adjustment. In order to overcome this limitation, the present invention adopts a semantic segmentation annotation method. In this method, the length and width of the minimum bounding rectangle of the segmentation annotation are denoted as h and w, respectively, where w represents the shorter side.

[0041] After adopting the segmentation annotation method, the target detection frame of the defect can be roughly annotated by calculating the minimum bounding rectangle of its segmentation annotation. If the defect area exceeds the preset threshold relative to the minimum bounding rectangle, and the short side w of the rectangle is less than 20 pixels, the present invention believes that the foreground information of the defect accounts for too high a proportion in the frame, so it is necessary to introduce more background information to achieve information balance. To this end, the detection frame will be adjusted accordingly to ensure that the model can learn and identify defect features more effectively.

[0042] When the aspect ratio w:h is less than 0.5, it indicates that the amount of information parallel to the short side is small, so the width w of the annotation box needs to be optimized. The amount by which the short side w needs to be increased is calculated using the long side h, a preset reasonable threshold, and the authority area. The adjustment process is to increase the width of the annotation box by adding an area of ​​the same size on both sides of the short side to ensure the balance of information in the box, thereby improving the accuracy of the model for defect detection. The updated short side is recorded as w'.

[0043] When the aspect ratio w:h is not less than 0.5, but the long side h is less than 15 pixels, it can be determined that the defect is a small-size defect. In response to this situation, the present invention proposes to synchronously update h and w to enhance the ability of the annotation box to capture small defects. The specific adjustment method is to update the size of the annotation box to three times the original size, that is, the new long side is 3h and the new short side is 3w. The increased area is evenly distributed on both sides of the original annotation box, and the increased size is equal. This not only retains the original information of the defect, but also introduces an appropriate amount of background context, which helps the model to more accurately identify small defects.

[0044] The present invention adopts a strategy of gradually increasing the frame size to optimize the adaptability and detection performance of the target detection frame when the aspect ratio w:h is not less than 0.5 and the long side h is not less than 15 pixels. The method expands the frame size by adding 2 pixels to the original short side w and long side h, then calculates the ratio of the defect area to the area of ​​the updated frame, and decides whether to continue adjusting based on the comparison result of the ratio with the preset threshold. If the ratio is less than the threshold, it indicates that the size of the frame has reached the ideal state, and the adjustment process is terminated immediately; if the ratio is still not less than the threshold, the operation of increasing the frame size is repeated until the ratio drops below the threshold, ensuring that the relative importance of the defect in the frame is maintained while introducing more background information.

[0045] Through the aforementioned strategy adjustment, the balanced distribution of foreground and background information in the annotation box is ensured, thereby providing optimized annotation data for model input to promote the model's effective recognition and learning of target objects.

[0046] 2. Optimize the strategy of adapting small images to model input

[0047] In industrial field environments, the size of collected images often far exceeds the input size required by the model. In order to reduce computational complexity and improve model performance, the present invention uses image cutting technology instead of direct scaling operations to retain the appearance characteristics of the original image to the greatest extent. During the image cutting process, it may be encountered that the size of the remaining part of the original image is smaller than the model input size. In this regard, the conventional processing method is to adopt the littlebox strategy, that is, to scale the image in proportion to the long side, and fill the short side with padding.

[0048] In the optimization scheme of the present invention, in order to ensure that the morphology of the defect is not changed, a more detailed processing method is adopted: padding is used on both the long and short sides, thereby avoiding the distortion of the defect size and other appearance features that may be caused by the scaling operation. This method not only retains the original features of the defect, but also ensures the accuracy of the model training data.

[0049] The above description only describes the specific implementation mode of the present invention. Various examples do not limit the essential content of the present invention. After reading the description, ordinary technicians in the relevant technical field can modify or deform the specific implementation modes described above without departing from the essence and scope of the invention.

Claims

1. A method for improving the quality of samples in target detection tasks, characterized by: The following steps are included: S1, obtain the segmentation annotation corresponding to the image; S2, calculating the pixel area of ​​the defective object in the annotation; S3, calculating the long side h and short side w of the minimum circumscribed rectangular box of the defect object; S4. Adjust the size of the annotation box according to the proportion of the defect area to the minimum circumscribed rectangular box; S5, obtaining the original image and the corresponding mask image; S6, traversing the original image to determine the image cutting position in a sliding window manner; S7. Process the original image and the mask image according to the cut position. If the cut-out image is smaller than the model size, expand the annotation frame around the small image to the model-fitting size and then input the model.

2. A method for improving the quality of target detection task samples as claimed in claim 1, characterized in that: In the step S4, if the proportion of the defect area to the minimum circumscribed rectangular frame exceeds a preset threshold, and the short side w of the rectangular frame is smaller than a preset pixel threshold, the marking frame is adjusted accordingly.

3. A method for improving the quality of target detection task samples as claimed in claim 1, characterized in that: In step S4, when the aspect ratio w:h is less than the preset ratio, only the width w of the annotation box is adjusted; the adjustment method is: using the long side h, a preset reasonable threshold and the authority area to calculate the amount of increase in the short side w.

4. A method for improving the quality of target detection task samples as claimed in claim 3, characterized in that: The specific adjustment process of the adjustment method is to add areas of the same size on both sides of the short side to expand the width of the marking box.

5. The method for improving the quality of target detection task samples according to claim 1, characterized in that: In step S4, when the aspect ratio w:h is not lower than the preset ratio, but the long side h is smaller than the set threshold, h and w are updated synchronously; the specific adjustment method is to update the size of the annotation box to three times the original size, that is, the new long side is 3h and the new short side is 3w.

6. A method for improving the quality of target detection task samples as claimed in claim 1, characterized in that: In the step S4, when the aspect ratio w:h is not less than a preset ratio and the long side h is not less than a set threshold, 2 pixels are added to the original short side w and long side h to expand the size of the annotation box.

7. A method for improving the quality of target detection task samples as claimed in claim 1, characterized in that: In the step S7, both the long side and the short side of the small image are expanded outward by padding.

Images