A Real-Time Target Detection Method for Micro-Defects on Strain Gauge Surfaces

Abstract

This invention relates to the fields of machine vision and industrial inspection, and discloses a real-time target detection method for micro-defects on strain gauge surfaces. It aims to address the problems of low detection accuracy and recall, insufficient generalization robustness, and difficulty in balancing real-time production line deployment and edge deployment caused by the small size of micro-defects, their tendency to be obscured by background, sample class imbalance, image style variations due to ambient lighting fluctuations, and high model complexity. The method includes: acquiring microscopic images; performing diverse processing on the original images to generate multi-style datasets; converting annotations to a standard target detection format and dividing the training set; performing sliding window slicing on large images, transforming coordinates to the local coordinate system of the slices, preserving complete targets, and controlling the ratio of positive to negative samples; training the model using a lightweight network; inputting the image to be tested for a single forward inference, outputting bounding boxes, categories, and confidence scores; performing threshold filtering and non-maximum suppression; measuring geometric parameters, determining the state according to multi-level thresholds, and outputting conclusions.

Description

Technical Field

[0001] This patent relates to the fields of machine vision inspection technology, artificial intelligence technology, and automated industrial inspection, specifically to a real-time target detection method for micro-defects on the surface of strain gauges. Background Technology

[0002] A strain gauge is a precision element used to measure strain, consisting of a sensitive grid, a substrate, and an adhesive. Similar to a flexible printed circuit board (FPC), it involves multiple manufacturing processes from design to warehousing. As an important detection method in non-electrical quantity measurement technology, strain gauges are widely used in various sensing devices for intelligent information detection, playing a crucial role in the intelligent and information-based development of industrial production. The surface quality of the strain gauge directly affects the measurement accuracy and reliability; therefore, the detection of surface defects is essential.

[0003] Because processes such as coating, photolithography, and etching can cause problems like burrs, depressions, broken grids, grid connections, and porosity in the sensitive grid of strain gauges, leading to their failure, each strain gauge must be rigorously inspected. The linewidth and spacing of the strain gauge's sensitive grid are only 1~20μm, requiring optical magnification to detect these micron-level defects.

[0004] Based on actual testing and analysis of a large number of strain gauge products, these micro-defects mainly include the following types: broken grid (cracks or broken grids on the strain gauge surface); bridged grid (grid bridging); deformed grid (grid deformation); metal residue / dirt (contaminants on the surface); contour defects (abnormal contours); dent (localized depressions on the grid surface); pinhole (tiny holes on the grid); bump (localized protrusions on the surface); and edgehole (holes in the pad area of ​​the strain gauge). These micro-defects are typically sub-millimeter in size, occupy a small proportion of the microscopic image, have low contrast, and their target area often accounts for less than 5% of the entire image, posing a significant technical challenge to automated inspection.

[0005] In actual industrial production environments, microscopic image acquisition systems face complex and variable lighting conditions. Different batches of products may exhibit variations in parameters such as brightness, contrast, and color temperature in the acquired images due to factors such as light source aging, power supply fluctuations, and changes in ambient light. Therefore, the detection system must be able to adapt to image input under different lighting conditions.

[0006] Traditional methods for detecting surface defects on strain gauges mainly fall into the following three categories:

[0007] The first method is the traditional manual visual inspection method. Inspectors use microscopes and other tools to visually inspect the surface of the strain gauges. This method has the following drawbacks: low inspection efficiency and long processing time; high error rate due to factors such as fatigue, memory errors, and lack of responsibility; short-term fluctuations in the strain gauge market demand, requiring skilled operators for manual inspection, making it difficult for supply to keep pace with market demand and easily leading to resource misallocation. More importantly, manual inspection cannot meet the high-speed inspection requirements of large-scale industrial production lines, presenting a significant bottleneck when dealing with mass production.

[0008] The second type is the traditional machine vision inspection method. This method is based on image processing technology and feature engineering. It extracts image features through algorithms such as edge detection (e.g., Canny operator, Harris corner detection), threshold segmentation (e.g., Otsu method), and morphological processing, and then uses a classifier to determine defects. However, traditional machine vision methods rely on fixed thresholds or manually designed features. When lighting conditions change, parameters need to be manually readjusted to ensure detection results. The linewidth of the sensitive grid on the strain gauge surface is only 1~20μm. Its micro-defects have low contrast in the image and are more sensitive to lighting fluctuations. Fixed thresholds are difficult to adapt to lighting changes in actual production environments.

[0009] The third type is detection methods based on deep learning. In recent years, deep learning methods have made significant progress in the field of image recognition. The YOLO series of algorithms, as classic algorithms in object detection, have continuously evolved since YOLOv1 was proposed in 2016: YOLOv2 introduced batch normalization and anchor boxes; YOLOv3 used multi-scale prediction and the Darknet-53 backbone network; YOLOv4 integrated techniques such as SPP, PANet, and Mosaic; YOLOv5 was implemented using PyTorch and its network structure was optimized; YOLOv7 introduced new modules such as ELAN; and YOLOv11 adopted an anchorless design and the DFL loss function. In addition, Faster R-CNN generates candidate boxes through a region proposal network, RetinaNet proposed Focal Loss to solve the class imbalance problem, CenterNet models object detection as a keypoint detection problem, and DETR utilizes the global modeling capabilities of the Transformer. Some researchers have attempted to apply deep learning to strain gauge defect detection, achieving good results in areas such as steel surface defect detection, PCB circuit board defect detection, and textile defect detection.

[0010] However, existing deep learning methods have the following problems:

[0011] I. Existing deep learning models mostly employ large backbone networks (such as ResNet, VGG, etc.), resulting in a huge number of parameters and inference computations. In the actual production line environment of strain gauges, the detection system needs to match the production line cycle for real-time response, but the huge computational overhead of general-purpose models makes it difficult to meet the real-time requirements of industrial scenarios;

[0012] Second, existing general-purpose object detection models (such as Faster R-CNN, SSD, etc.) are mainly designed for natural image datasets such as ImageNet. Their anchor mechanism and feature pyramid structure have limited recall for small objects. Strain gauge micro-defects typically account for less than 5% of the area and the sensitive grid linewidth is only 1~20μm. The effective pixels in the shallow feature maps of the model are very few, making it difficult to form effective feature representation, resulting in a high false negative rate for small objects.

[0013] Third, the model is large in size and difficult to deploy on edge devices or embedded systems;

[0014] Fourth, the training data has a class imbalance problem. The number of samples of common defects such as broken wires and metal residues may be more than ten times that of rare defects such as wire pinholes and corrosion deformation, causing the model to over-focus on the majority class samples.

[0015] Fifth, existing methods often neglect the importance of performing diverse preprocessing on training data to improve model robustness, resulting in trained models that are more sensitive to changes in image style.

[0016] In summary, existing methods for detecting surface defects on strain gauges have shortcomings in terms of detection accuracy, real-time performance, edge deployment capability, class imbalance handling, and model robustness. There is an urgent need for a strain gauge surface micro-defect detection technology that can simultaneously meet the requirements of high accuracy, high real-time performance, easy edge deployment, and strong robustness.

[0017] Furthermore, there is currently a lack of unified industry standards for the industrial inspection of micro-defects on the surface of strain gauges, and different companies have different requirements for defect tolerance. Based on in-depth research into the current state of the strain gauge market and the actual needs of users, this invention, combined with the typical structural dimensions of the strain gauge sensitive grid (wire width 1~20μm) and operating conditions, has independently developed a differentiated defect judgment threshold system, providing a quantitative basis for the surface quality inspection of strain gauges. Summary of the Invention

[0018] The purpose of this invention is to solve the technical problems caused by the extremely small size of micro-defects in strain gauges, which are easily submerged by the background; the severe imbalance of defect sample categories; the variation of image styles due to the fluctuation of lighting in industrial sites; and the high computational complexity of existing detection models. These problems result in low accuracy and recall of micro-defect detection, insufficient model generalization robustness, and difficulty in meeting the real-time requirements of industrial production lines and the deployment requirements of edge devices.

[0019] To achieve the above objectives, the present invention employs the following technical means:

[0020] This invention provides a real-time target detection method for micro-defects on the surface of strain gauges, comprising the following steps:

[0021] Step 1, Microscopic Image Acquisition: Acquire images of the strain gauge surface using a microscopic imaging device;

[0022] Step 2, Multi-style dataset construction and preprocessing: Perform diverse image processing on the original image set to generate an augmented dataset containing different visual styles, and merge the original image set and the augmented dataset to obtain the training dataset;

[0023] Step 3, Dataset Format Conversion and Division: Convert the annotation format of the training dataset into the standard object detection format, and divide it into training set, validation set and test set according to a preset ratio;

[0024] Step 4: Preprocessing of sliding window slices for microscopic images: Perform sliding window slicing on large-size microscopic images, transform the coordinate system of the original image to the local coordinate system of the slice, retain only the target labels that fall completely into the slice, and perform positive and negative sample ratio control during the slicing process;

[0025] Step 5: Lightweight deep learning model training: Using the YOLO series of lightweight target detection models, network training is performed based on the training set to obtain the strain gauge micro-defect detection model;

[0026] Step 6: Real-time defect target detection reasoning: Input the strain gauge microscopic image to be detected into the strain gauge micro-defect detection model, perform a single forward propagation reasoning, and output the bounding box coordinates, defect category and confidence level of the defect target;

[0027] Step 7, Post-processing of detection results: Confidence threshold filtering and non-maximum suppression are performed on the inference results, overlapping detection boxes are removed, and defect classification statistics and visualization output are completed;

[0028] Step 8: Industrial Standard Defect Judgment: Through thorough market research on strain gauges and in combination with the actual application needs of strain gauge users, differentiated defect judgment thresholds are established for some defects. The geometric parameters of the detected defect targets are accurately measured. Based on the preset multi-level industrial judgment thresholds, qualified / unqualified quantitative judgment is made, and the judgment conclusion is output.

[0029] In the above scheme, step 2 specifically includes:

[0030] White balance control and image adjustment are performed on the images in the original image set to generate a white balance color image dataset;

[0031] The images in the original image set are subjected to white balance control, image adjustment, grayscale conversion and median filtering to generate a black and white image dataset;

[0032] Merge the original image set, the white balance color image dataset, and the black and white image dataset, rename the files using consecutive numerical numbers, and update the path field of the corresponding annotation files;

[0033] The grayscale conversion uses the following formula:

[0034]

[0035] in, The value represents the grayscale value, and R, G, and B represent the pixel values ​​of the red, green, and blue channels, respectively.

[0036] In the above scheme, step 3 uses the following coordinate mapping formula for format conversion:

[0037]

[0038] Where W and H are the width and height of the image, respectively. )and( ) represents the coordinates of the top-left and bottom-right corners of the bounding box in VOC format. ) represents the normalized center point coordinates, and w and h represent the normalized bounding box width and height, both ranging from [0, 1].

[0039] The dataset is divided into three parts: training set (81%), validation set (9%), and test set (10%).

[0040] In the above scheme, step 4 specifically includes the following sub-steps:

[0041] Step 4.1, Sliding window slicing execution: Set the slicing parameters according to the preset slice size and overlap rate, and use the step size to perform sliding window slicing on the large-size microscopic image. For each slice position in the original image, obtain the slice image at that position.

[0042] Step 4.2, Coordinate system transformation: Transform the annotations in the original image coordinate system to the local coordinate system of the slice. Perform coordinate transformation according to the starting position and size of the slice to obtain the normalized coordinates in the local coordinate system of the slice.

[0043] Step 4.3, Annotation Filtering: Only retain the annotations of targets that completely fall within the slice for subsequent training;

[0044] Step 4.4, First stage information content sampling: Calculate the information content score for each negative sample slice without defect annotation, and linearly map the information content score to the probability interval [0.05, 1.0]. Based on this probability, randomly decide whether to retain the negative sample slice. The negative sample window with the smallest information content has a 5% survival chance.

[0045] Fractions are calculated using the following formula:

[0046]

[0047] in, Let V be the variance of the grayscale image. and The horizontal and vertical gradients are calculated by the Sobel operator, respectively. Total number of pixels;

[0048] Step 4.5, Second Stage Global Proportional Gate: Control the total number of negative samples based on the number of saved positive and negative sample slices. When the total number of negative samples exceeds the target value, randomly discard the excess negative sample slices proportionally.

[0049] The target negative sample count is calculated using the following formula:

[0050]

[0051] in, Let V be the variance of the grayscale image. and The horizontal and vertical gradients are calculated by the Sobel operator, respectively. Total number of pixels; The target number of negative samples. To minimize the number of negative samples to retain, This represents the upper limit of the proportion of negative and positive samples. This represents the number of positive sample slices that have been saved.

[0052] The synergistic effect of multi-style dataset construction in step 2: Existing strain gauge defect detection methods mostly use single-style training data. When industrial lighting conditions change, the acquired images differ in brightness, contrast, and color temperature, causing the features learned by the model on a single data set to fail to generalize effectively. Specifically, if only white-balanced color images are used for training, the model may over-rely on color features and perform poorly on grayscale images; if only black and white images are used for training, the defect discrimination information provided by the color channels is lost. This invention innovatively adopts a synergistic combination of white-balanced color images and black and white images: white balance processing and brightness / contrast adjustment enhance the contrast between the strain gauge sensitive grid area and the background, making it easier for the model to learn the edge and texture features of defects; the grayscale converted image forces the model to learn color-independent feature representations, enhancing the model's adaptability to different imaging modes. The synergistic combination of these two approaches enables the model to learn defect features from both color and texture dimensions simultaneously, significantly improving generalization robustness under varying lighting conditions and breaking through the generalization bottleneck of single-data style training.

[0053] The synergistic effect of sliding window slicing and positive / negative sample ratio control in step 4: Existing large-size image detection methods typically downsample the original image before inputting it into the model. However, the area ratio of micro-defects in strain gauges is less than 5%, and downsampling leads to a significant loss of defect detail information. Specifically, if only sliding window slicing is used without controlling the positive / negative sample ratio, a large number of defect-free negative sample slices will cause the model to over-focus on background areas, reducing the defect detection rate. If only random sampling is used to control negative samples, background samples with texture information may be lost, resulting in insufficient model discrimination ability for complex backgrounds. This invention innovatively adopts a synergistic combination of sliding window slicing and two-stage negative sample screening: sliding window slicing retains the defect details at the original resolution, solving the problem of small target information loss caused by downsampling; the information sampling in the two-stage screening mechanism ensures that negative sample slices with visual information value are retained, preventing the model from overfitting to a fixed background pattern. At the same time, the global ratio gate limits the ratio of negative and positive samples within a preset range, balancing defect feature learning and background discrimination ability improvement. The synergistic combination of the two effectively improves the training efficiency and generalization performance of the model while ensuring the accuracy of small target detection.

[0054] Synergistic effect of steps 2 and 4: Existing micro-defect detection methods rarely consider the matching between slice size and defect scale during data preprocessing, resulting in small targets having too small a proportion in the model input after slicing, making it difficult to form effective feature responses. Specifically, if only multi-style data augmentation is performed without considering slice size, small targets after slicing may shrink further or even disappear; if only a fixed slice size is used without considering data augmentation, the variation in defect contrast under different lighting conditions will exacerbate the difficulty of identifying small targets. This invention innovatively adopts a synergistic combination of multi-style data construction and sliding window slice preprocessing: multi-style data augmentation improves the contrast between defects and background, making small targets easier for the model to capture in the slice; the preset slice size ensures that small targets maintain a sufficient number of pixels in the model input, and the preset overlap rate ensures complete capture of defect targets. The synergistic combination of these two methods jointly guarantees the detection accuracy of small targets from both data quality and input size dimensions.

[0055] In the above scheme, step 5 is based on the multi-style dataset constructed in step 2 and the slice data processed in step 4. The YOLO series lightweight target detection model is used for training to achieve effective detection of strain gauge micro-defects with an area ratio of less than 5%. During training, the input image size is uniformly adjusted to 640×640 pixels, the training rounds are set to 500 rounds, the batch size is set to 128, the optimizer adopts the stochastic gradient descent algorithm, the initial learning rate is set to 0.01, and mixed precision training is enabled.

[0056] In the above scheme, in step 6, the micro-defect categories identified by the model include 9 categories: broken grid, connected grid, deformation, metal residue / dirt, contour defects, dents, wire mesh pinholes, protrusions, and pad vias; the inference process adopts a single-stage detection method, which completes target classification and bounding box regression simultaneously in one forward propagation.

[0057] In the above scheme, step 7 specifically includes: setting the confidence threshold to 0.1, retaining only the detection results with a confidence level higher than the threshold; setting the cross-union ratio threshold for non-maximum suppression to 0.5; performing classification statistics according to the defect category, and drawing the detection bounding box, labeling the defect category and confidence value on the original image.

[0058] In the above scheme, the industrial standard defect determination in step 8 includes:

[0059] Protrusion defects: The strict standard is that a protrusion height exceeding one-third of the wire grid spacing is considered unqualified, while the lenient standard is that a protrusion height exceeding two-thirds of the wire grid width is considered unqualified.

[0060] Dirt defects and pinhole defects: The strict standard is that if the defect area exceeds half of the wire mesh end area, it is considered unqualified. The lenient standard is that if the defect area is between half and the full size and the number is more than three, it is considered unqualified. Alternatively, if the defect area exceeds the full size, it is directly considered unqualified.

[0061] In the above scheme, in step 1, the microscopic imaging device includes a UVC protocol-compatible microscopic camera with a USB interface and an adjustable light source module; the microscopic camera supports multiple resolution settings, the exposure parameter adjustment range is -12 to 0, the brightness parameter and contrast parameter adjustment range are both 0 to 255, and the frame rate parameter adjustment range is 1 to 120fps.

[0062] The above scheme also includes the step of exporting the trained strain gauge micro-defect detection model into the ONNX standardized format for use in edge computing device deployment; the sliding window slice size in step 4 is set to 640×640 pixels, and the overlapping pixels of adjacent slices are set to 160 pixels.

[0063] Because the present invention employs the above-mentioned technical means, it has the following beneficial effects:

[0064] 1. This invention employs a multi-style dataset construction strategy, introducing training samples of various visual styles, such as white-balanced color images and black-and-white images, during the training phase. This significantly enhances the model's adaptability to different lighting conditions and imaging modes. While the image processing methods described above are conventional, the innovation of this invention lies in: considering the low proportion (less than 5%) of micro-defects on the strain gauge surface and the sensitivity grid linewidth of only 1-20 μm in the image, a synergistic combination of white balance processing and multi-parameter joint adjustment is used to enhance the contrast between the strain gauge's sensitivity grid area and the background, making it easier for the model to learn the edge and texture features of defects; a synergistic combination of grayscale conversion and median filtering forces the model to learn color-independent feature representations, enhancing the model's adaptability to different imaging modes. If only single-style data is used for training, the model's performance will degrade under varying lighting conditions; without multi-style data construction, the model struggles to cope with the complex lighting fluctuations in industrial environments. This invention, through the aforementioned specific combination, enables the model to learn defect features simultaneously from both color and texture dimensions, effectively improving generalization robustness under complex lighting conditions in industrial environments.

[0065] 2. The YOLO series model used in this invention is a single-stage target detection algorithm, which completes target classification and localization in one forward propagation, resulting in high inference efficiency. Combined with lightweight model selection (YOLO11 series) and edge deployment strategy, it can meet the real-time requirements of industrial online inspection scenarios and satisfy the real-time inspection needs of industrial production lines.

[0066] 3. This invention employs a sliding window slicing preprocessing strategy to avoid the resolution reduction caused by scaling the original image, thus preserving the micro-defect details of the strain gauge sensitive grid. While maintaining the high resolution of the original image, the large-size microscopic image is divided into 640×640 pixel sub-image blocks, enabling the model to learn micro-defect features at the original resolution level, effectively solving the problem of insufficient defect detection accuracy in large-size images. Simultaneously, by setting a 160-pixel slice overlap rate, it ensures that the defect target is not truncated by the slice boundaries, guaranteeing the integrity of the detection.

[0067] 4. This invention achieves positive-to-negative sample ratio control through a two-stage negative sample screening mechanism, effectively solving the problem of reduced training efficiency caused by an excessive number of negative samples during sliding window slicing. The first stage of information sampling ensures that negative sample slices retain visual information value, preventing the model from overfitting to a fixed background pattern; the second stage of global ratio gate limits the ratio of negative to positive samples to within 2:1, balancing defect feature learning and background discrimination ability, significantly improving model training efficiency and detection performance.

[0068] 5. This invention can simultaneously detect nine different types of micro-defects on the surface of strain gauges, including broken gauges, bridged gauges, deformed gauges, metal residue / dirt, contour defects, dents, wire mesh pinholes, bumps, and edgeholes. Comprehensive defect information can be obtained in a single inspection, improving detection efficiency and reducing the risk of missed defects.

[0069] 6. This invention supports exporting the trained model to the ONNX standardized format. The model can run on various edge computing platforms without relying on cloud servers, realizing offline deployment and edge computing of the detection system, effectively reducing data transmission latency and system bandwidth requirements, and improving applicability in industrial settings.

[0070] 7. Explanation of the synergistic effects between the steps of this invention:

[0071] 7.1 Collaboration of White Balance Color Images and Black and White Images in Step 2: Existing strain gauge defect detection methods mostly use single-style training data, resulting in insufficient model generalization ability when industrial lighting conditions change. If only white balance color images are used, the model may over-rely on color features; if only black and white images are used, defect discrimination information in the color channels is lost. This invention, through a specific combination of white balance processing and grayscale conversion, enables the model to learn defect features from both color and texture dimensions simultaneously, breaking through the generalization bottleneck of a single data style.

[0072] 7.2 Synergy between sliding window slicing and positive / negative sample control in step 4: Existing large-size image detection methods typically involve direct downsampling, leading to the loss of micro-defect details in strain gauges. If only slicing is used without controlling the sample ratio, too many negative samples will reduce training efficiency; if only random sampling is used, important background features may be lost. This invention preserves small target details through slicing and controls the sample ratio through two-stage screening, balancing detection accuracy and training efficiency.

[0073] 7.3 Synergy between Steps 2 and 4: Multi-style data augmentation improves defect contrast, making small targets easier for the model to capture; slice size ensures that small targets retain sufficient pixels in the input. Both steps, from the perspectives of data quality and input size, jointly guarantee the accuracy of small target detection.

[0074] 7.4 Synergy of Steps 2, 4 and 5: Multi-style data, slice processing and YOLO lightweight model are combined to realize a complete technical solution for strain gauge micro-defect detection, breaking through the performance limit of a single method in this scenario. Attached Figure Description

[0075] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0076] Figure 1 This is an overall flowchart of the method of the present invention, which shows the complete process from microscopic image acquisition to detection result output.

[0077] Figure 2 These are images showing the preprocessing results of a dataset with different styles. They demonstrate the different styles of images generated from the original image after steps such as white balance processing, image adjustment, grayscale conversion, and median filtering.

[0078] Figure 3 These are example images of different defect types, demonstrating the detection effect of the method of the present invention on nine different types of defects. Detailed Implementation

[0079] The embodiments of the present invention will be described in detail below. Although the present invention will be described and illustrated in conjunction with some specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, any modifications or equivalent substitutions made to the present invention should be covered within the scope of the claims of the present invention.

[0080] Furthermore, to better illustrate the present invention, numerous specific details are set forth in the following detailed embodiments. Those skilled in the art will understand that the present invention can be practiced without these specific details.

[0081] The purpose of this invention is to solve the following technical problems existing in current methods for detecting micro-defects on the surface of strain gauges:

[0082] 1. Small Target Defect Detection Problem: Surface defects on strain gauges are typically small in size, with sensitive grid linewidths of only 1~20μm. They often occupy less than 5% of the area in the entire image, making them easily obscured by complex background information. Traditional target detection algorithms tend to suffer from insufficient feature representation when processing small targets, resulting in low detection recall.

[0083] 2. Class Imbalance Issue: In actual production processes, the frequency of different types of defects varies significantly. For example, the number of samples for common defects such as broken grids and metal residues may be more than ten times that of rare defects such as wire grid pinholes and corrosion deformation. This class imbalance can cause the model to overemphasize majority class samples during training, while underlearning minority class samples, ultimately affecting the overall detection performance.

[0084] 3. The challenge of balancing real-time performance and accuracy: Industrial production lines have stringent requirements for the real-time response capability of detection systems, typically needing to complete the detection of a single image within 100ms. However, high-precision deep learning models often have high computational complexity, with inference times potentially reaching hundreds of milliseconds or even longer.

[0085] 4. Image style change adaptability issues: Industrial sites have factors such as light source fluctuations, power instability, and changes in ambient light, which may cause differences in brightness, contrast, color temperature, etc. in the acquired images.

[0086] To address the aforementioned technical issues, this invention combines "dual-style (white balance color and grayscale filtering) data construction" with "sliding window slicing and two-stage negative sample ratio control based on information content scoring" in a specific logical combination, and adapts it to a lightweight YOLO model to form a "data-sampling-model" closed-loop detection scheme for strain gauge micro-defects with an area ratio of <5%.

[0087] To facilitate a better understanding of the core innovations of this invention by those skilled in the art, the following explanation is provided regarding the technical considerations underlying the adoption of this invention's technical solutions:

[0088] If this invention only uses conventional multi-style enhancement or single lighting adaptation processing, the model is very likely to fail due to over-reliance on color / brightness features when the lighting fluctuates in industrial settings. If only large-size micrographs are directly downsampled or conventional sliding window slicing is used, although it can match the model input size, it will cause severe dilution of feature pixels of sub-millimeter micro-defects. In addition, the massive amount of blank / simple background negative samples generated by slicing will dominate the loss function calculation, causing the model to miss defects.

[0089] This invention produces a non-obvious synergistic effect through the above specific combination: dual-style processing (color enhancement contrast, grayscale forced decoupling of color dependence) pre-"highlights" the texture and edge essential features of micro-defects before data input; sliding window slicing retains the physical pixels of defects at the original resolution with a fixed overlap rate, avoiding the disappearance of small targets caused by downsampling; and the two-stage negative sample control is not simply random discarding, but calculates the slice information content score through variance and Sobel gradient, linearly maps it to probability for random survival determination (ensuring a minimum of 5% basic background retention to prevent overfitting to a single complex background), and then precisely controls the upper limit of negative and positive samples through a global proportional gate.

[0090] Specifically, the invention's style enhancements address the issue of "feature drift caused by varying illumination," slicing solves the problem of "small target annihilation due to resolution loss," and two-stage sampling precisely removes "redundant and invalid background" while retaining "high-information interference background," thus forcing the limited computing power of lightweight YOLO to focus on the true micro-defect discrimination. This combination enables the system to simultaneously overcome the performance bottlenecks of "high recall rate for extremely small targets" and "strong robustness to dramatic illumination changes" under computing power constraints, even under edge deployment conditions.

[0091] This invention provides a real-time target detection method for micro-defects on the surface of strain gauges, used to detect micro-defects with a linewidth of only 1~20μm in the sensitive grid region of the strain gauge, comprising the following steps:

[0092] Step 1: Microscopic Image Acquisition: Images of the strain gauge surface are acquired using a microscopic imaging device. This device includes a USB-interface UVC protocol-compatible microscope camera and an adjustable light source module. The microscope camera supports multiple resolution settings, including 640×480, 1280×720, 1920×1080, 2560×1440, and 3840×2160. The camera's exposure parameters are adjustable from -12 to 0, brightness parameters from 0 to 255, contrast parameters from 0 to 255, and frame rate parameters from 1 to 120fps. During acquisition, the camera's operating status information, including the current frame rate and image resolution, is displayed in real time.

[0093] Step 2: Multi-style Dataset Construction and Preprocessing: This invention designs a multi-style dataset construction strategy. By performing diverse image processing on the original dataset, training samples containing different visual styles are generated, thereby significantly improving the model's adaptability to different lighting conditions and imaging modes. Specifically, the processing flow includes the following:

[0094] Step 2.1: Raw Dataset Preparation: The acquired strain gauge microscopic images are labeled in VOC format. The labeling includes the bounding box coordinates and defect category for each defect in the image. The VOC format XML annotation file contains the image's filename, path, and size information, as well as the name, difficulty tag, and bounding box coordinates for each object. The raw dataset is used as the base dataset and is denoted as the raw image set.

[0095] Step 2.2, White Balance Color Image Generation: White balance processing and image adjustment are performed on each image in the original image set to generate the first enhanced dataset. Specific processing steps include: White Balance Control: Gain adjustment is applied to the RGB channels of the image. Each channel is weighted based on preset red, green, and blue gain parameters. Simultaneously, color temperature adjustment is performed. The red and blue channels are compensated based on the difference between the current color temperature and the reference color temperature. Image Adjustment: Brightness, contrast, saturation, and sharpness are adjusted sequentially on the white balance processed image. Wide Dynamic Range Processing: The L channel of the image is dynamically expanded according to preset parameters. The processed image is saved as a white balance color image, generating the first enhanced dataset.

[0096] Step 2.3, Black and White Image Generation: Perform a complete image processing workflow on each image in the original image set to generate the second enhanced dataset. Specific processing steps include: First, perform white balance control, using the same parameters as in Step 2.2. Then, perform image adjustment, using the same parameters as in Step 2.2. Next, perform grayscale conversion, converting the processed color image to a grayscale image using the following formula:

[0097]

[0098] Where Gray represents the grayscale value, and R, G, and B represent the pixel values ​​of the red, green, and blue channels, respectively.

[0099] Finally, median filtering is performed, using a median filter with a kernel size of 3 to denoise the grayscale image and eliminate salt-and-pepper noise generated during image acquisition.

[0100] The images that have undergone the complete processing are saved as black and white images to generate a second augmented dataset.

[0101] Step 2.4, Dataset Merging: The original image set, the first augmentation dataset, and the second augmentation dataset are merged to generate the final training dataset. The specific merging method is as follows: images and corresponding annotation files from the three datasets are copied to a unified output directory. The merged files are renamed using consecutive five-digit numbering. Simultaneously, the filename and path fields in the annotation XML files are updated to match the new filenames. The merged dataset is three times the size of the original dataset, effectively expanding the diversity of the training samples.

[0102] Step 3: Dataset Format Conversion and Partitioning: Convert the merged VOC format labeled data to YOLO format labeled data. The specific conversion method is as follows: For the bounding box coordinates in the VOC format... , , , Calculate the center point coordinates and dimensions (width and height) in YOLO format. The coordinate transformation uses the following formula:

[0103]

[0104]

[0105]

[0106]

[0107] Where W and H are the width and height of the image, respectively. )and( ) represents the coordinates of the top-left and bottom-right corners of the bounding box in VOC format. ) represents the normalized center point coordinates, and w and h represent the normalized bounding box width and height, both ranging from [0,1]. The dataset is divided as follows: the training set accounts for 81% of the total dataset, the validation set accounts for 9% of the total dataset, and the test set accounts for 10% of the total dataset.

[0108] Step 4: Microscopic Image Sliding Slicing Preprocessing: Large-size microscopic images are processed using sliding window slicing. Segmentation parameters are set according to preset slice size and overlap rate, and the image is sliced ​​using a step size. For each slice location in the original image, the slice image at that location is obtained, and the annotations in the original image coordinate system are transformed to the slice's local coordinate system. Coordinate transformation is performed based on the slice's starting position and size to obtain normalized coordinates in the slice's local coordinate system. Only the annotations of targets completely falling within the slice are retained for subsequent training.

[0109] Simultaneously, a two-stage screening process is implemented for negative sample slices to control the ratio of positive to negative samples: During the sliding window slicing process, a two-stage screening process is performed for negative sample slices to control the ratio of positive to negative samples. The specific screening mechanism is as follows: The first stage is information content sampling: For each negative sample slice without defect annotations, an information content score is calculated using the following formula:

[0110]

[0111] in, Let V be the variance of the grayscale image. and The horizontal and vertical gradients are calculated by the Sobel operator, respectively. The total number of pixels. The information score is linearly mapped to a probability range of [0.05, 1.0]. Based on this probability, it is randomly decided whether to retain the negative sample slice. The negative sample window with the lowest information score still has a 5% survival chance to prevent overfitting to a fixed background pattern. The second stage is a global proportional gate: based on the number of saved positive sample slices and the number of negative sample slices, the total number of negative samples is controlled. The formula for calculating the target number of negative samples is:

[0112]

[0113] in, To minimize the number of negative samples to retain, This represents the upper limit of the proportion of negative and positive samples. This represents the number of positive sample slices already saved. When the total number of negative samples exceeds the target value, the excess negative sample slices are randomly discarded proportionally. This mechanism effectively balances defect feature learning and background discrimination improvement, avoiding reduced training efficiency or excessive model focus on defect-free regions due to an excessive number of negative samples.

[0114] Step 5: Lightweight Deep Learning Model Training: The YOLO series of lightweight object detection models are used for training. The YOLO series includes five different model sizes: YOLO11n is nanoscale, with approximately 3.2 million model parameters and approximately 8.7 GFLOPs of floating-point operations, suitable for edge devices and scenarios requiring extreme speed; YOLO11s is small, with approximately 11.2 million model parameters and approximately 28.6 GFLOPs of floating-point operations, suitable for scenarios requiring a balance between speed and accuracy; YOLO11m is medium-sized, with approximately 25.9 million model parameters and approximately 78.9 GFLOPs of floating-point operations, suitable for scenarios requiring high detection accuracy; YOLO11l is large, with approximately 43.7 million model parameters and approximately 165.2 GFLOPs of floating-point operations, suitable for scenarios requiring high accuracy; and YOLO11x is ultra-large, with approximately 68.2 million model parameters and approximately 257.8 GFLOPs of floating-point operations, suitable for scenarios requiring the highest accuracy. The input images for model training were uniformly resized to 640 x 640 pixels, scaled while maintaining the aspect ratio, and filled with gray edges. The training epochs were set to 500, the batch size to 128, the optimizer to use stochastic gradient descent, the initial learning rate to 0.01, the early stopping patience value to 100 epochs to prevent overfitting, and mixed precision training to accelerate the training process.

[0115] Step 6: Real-time Defect Target Detection Inference: The trained YOLO model is used for real-time inference on the strain gauge microscopic images to be detected. The model can identify nine types of micro-defects on the strain gauge surface, including broken, bridged, deformed, metal residue / dirt, contour defects, dents, pinholes, bumps, and edgeholes. The model outputs the bounding box coordinates, defect category number, and confidence score for each detected defect target. The inference process adopts a single-stage detection method, completing target classification and bounding box regression simultaneously in one forward propagation, ensuring real-time detection.

[0116] Step 7: Post-processing of detection results: Post-process the detection results output by the model. Specifically, this includes: First, setting a confidence threshold, which is 0.1 by default, and only retaining detection results with a confidence level higher than this threshold to filter out low-quality detection boxes; then, using a non-maximum suppression algorithm to remove overlapping detection boxes, with the intersection-union ratio (IUU) threshold set to 0.5; finally, classifying and statistically analyzing the detected defect categories, and visualizing the detection results, including drawing detection bounding boxes on the original image, labeling defect categories, and assigning confidence scores.

[0117] Step 8: Industry Standard Defect Judgment: The defects detected by the deep learning model are precisely measured and quantitatively analyzed. Based on the industry standards of the strain gauge user, a pass / fail judgment is made. Strain gauge users have varying precision requirements for different types of defects. This invention uses algorithms to precisely measure defect parameters, overcoming the limitations of traditional rough judgments based on visual inspection. Specifically, it includes the following processing steps:

[0118] Step 8.1, Precise Measurement of Defect Parameters: For each detected defect target, precise geometric parameter measurements are performed based on its bounding box coordinates and pixel area. In this embodiment, for a protrusion defect, the vertical height of the protrusion area is measured; for a contamination defect, the pixel area of ​​the contamination area is measured; and for a pad via defect, the pixel area of ​​the via area is measured.

[0119] Step 8.2, Industry Standard Judgment: Defect parameters are judged according to the industry standards preset by the strain gauge user. Multiple judgment thresholds are set for different application scenarios' quality requirements, including both strict and lenient standard modes.

[0120] (a) Criteria for judging protrusion defects: Based on the technical survey results of multiple strain gauge users, the strict standard is that a protrusion height exceeding one-third of the wire grid spacing is judged as unqualified; the lenient standard is that a protrusion height exceeding two-thirds of the wire grid width is judged as unqualified.

[0121] (II) Criteria for judging contamination defects: Based on the technical survey results of multiple strain gauge users, the strict standard is that when the contamination area exceeds half of the wire grid end area, it is judged as unqualified; the lenient standard is that when the contamination area is between half of the wire grid end area and the complete size and the number is more than three, it is judged as unqualified; when the contamination area exceeds the complete size of the wire grid end, it is directly judged as unqualified.

[0122] (III) Criteria for judging pinhole defects in pads: Based on the technical survey results of multiple strain gauge users, the strict standard is that the pinhole area of ​​the pad exceeds half of the wire grid end area and is judged as unqualified; the lenient standard is that the pinhole area of ​​the pad is between half of the wire grid end area and the full size and the number is more than three and is judged as unqualified; the pinhole area of ​​the pad exceeds the full size of the wire grid end and is directly judged as unqualified.

[0123] The judgment thresholds were independently formulated by this invention based on extensive market research in the strain gauge industry. Since there is currently a lack of unified national or industry standards for the detection of micro-defects on the surface of strain gauges, this invention established the above-mentioned multi-level judgment criteria by surveying the actual quality requirements of strain gauge manufacturers and users, and combining the typical structural dimensions of the sensitive grid.

[0124] Step 8.3: Output Judgment Conclusion: The strain gauge under test is automatically judged as qualified or unqualified according to the preset judgment criteria. In the embodiment, when all detected defects meet the qualification conditions, the strain gauge is output as qualified; when any defect meets the unqualification conditions, the strain gauge is output as unqualified. Finally, a detailed test report is output, including the type and quantity of detected defects, the accurate measurement value of each defect and the judgment result, as well as the final qualified / unqualified conclusion.

Claims

1. A real-time target detection method for micro-defects on the surface of strain gauges, characterized in that, Includes the following steps: Step 1, Microscopic Image Acquisition: Acquire images of the strain gauge surface using a microscopic imaging device; Step 2, Multi-style dataset construction and preprocessing: Perform diverse image processing on the original image set to generate an augmented dataset containing different visual styles, and merge the original image set and the augmented dataset to obtain the training dataset; Step 3, Dataset Format Conversion and Division: Convert the annotation format of the training dataset into the standard object detection format, and divide it into training set, validation set and test set according to a preset ratio; Step 4: Preprocessing of sliding window slices for microscopic images: Perform sliding window slicing on large-size microscopic images, transform the coordinate system of the original image to the local coordinate system of the slice, retain only the target labels that fall completely into the slice, and perform positive and negative sample ratio control during the slicing process; Step 5: Lightweight deep learning model training: Using the YOLO series of lightweight target detection models, network training is performed based on the training set to obtain the strain gauge micro-defect detection model; Step 6: Real-time defect target detection reasoning: Input the strain gauge microscopic image to be detected into the strain gauge micro-defect detection model, perform a single forward propagation reasoning, and output the bounding box coordinates, defect category and confidence level of the defect target; Step 7, Post-processing of detection results: Confidence threshold filtering and non-maximum suppression are performed on the inference results, overlapping detection boxes are removed, and defect classification statistics and visualization output are completed; Step 8, Industrial Standard Defect Judgment: Accurately measure the geometric parameters of the detected defective target, make a qualified / unqualified quantitative judgment based on the preset multi-level industrial judgment thresholds, and output the judgment conclusion.

2. The method according to claim 1, characterized in that, Step 2 specifically includes: White balance control and image adjustment are performed on the images in the original image set to generate a white balance color image dataset; The images in the original image set are subjected to white balance control, image adjustment, grayscale conversion and median filtering to generate a black and white image dataset; Merge the original image set, the white balance color image dataset, and the black and white image dataset, rename the files using consecutive numerical numbers, and update the path field of the corresponding annotation files; The grayscale conversion uses the following formula: in, R represents the grayscale value, and R, G, and B represent the pixel values ​​of the red, green, and blue channels, respectively.

3. The method according to claim 1, characterized in that, In step 3, the format conversion uses the following coordinate mapping formula: Where W and H are the width and height of the image, respectively. )and( ) represents the coordinates of the top-left and bottom-right corners of the bounding box in VOC format. ) represents the normalized center point coordinates, and w and h represent the normalized bounding box width and height, both ranging from [0,1]. The dataset is divided into three parts: training set (81%), validation set (9%), and test set (10%).

4. The method according to claim 1, characterized in that, Step 4 specifically includes the following sub-steps: Step 4.1, Sliding window slicing execution: Set the slicing parameters according to the preset slice size and overlap rate, and use the step size to perform sliding window slicing on the large-size microscopic image. For each slice position in the original image, obtain the slice image at that position. Step 4.2, Coordinate system transformation: Transform the annotations in the original image coordinate system to the local coordinate system of the slice. Perform coordinate transformation according to the starting position and size of the slice to obtain the normalized coordinates in the local coordinate system of the slice. Step 4.3, Annotation Filtering: Only retain the annotations of targets that completely fall within the slice for subsequent training; Step 4.4, First stage information content sampling: Calculate the information content score for each negative sample slice without defect annotation, and linearly map the information content score to the probability interval [0.05, 1.0]. Based on this probability, randomly decide whether to retain the negative sample slice. The negative sample window with the smallest information content has a 5% survival chance. Fractions are calculated using the following formula: in, Let V be the variance of the grayscale image. and The horizontal and vertical gradients are calculated by the Sobel operator, respectively. Total number of pixels; Step 4.5, Second Stage Global Proportional Gate: Control the total number of negative samples based on the number of saved positive and negative sample slices. When the total number of negative samples exceeds the target value, randomly discard the excess negative sample slices proportionally. The target negative sample count is calculated using the following formula: in, The target number of negative samples. To minimize the number of negative samples to retain, This represents the upper limit of the proportion of negative and positive samples. This represents the number of positive sample slices that have been saved.

5. The method according to claim 1, characterized in that, Step 5, based on the multi-style dataset constructed in step 2 and the slice data processed in step 4, uses a lightweight target detection model from the YOLO series for training to achieve effective detection of micro-defects in strain gauges with an area ratio of less than 5%. During training, the input image size is uniformly adjusted to 640×640 pixels, the training rounds are set to 500 rounds, the batch size is set to 128, the optimizer uses the stochastic gradient descent algorithm, the initial learning rate is set to 0.01, and mixed precision training is enabled.

6. The method according to claim 1, characterized in that, In step 6, the micro-defect categories identified by the model include 9 categories: broken grid, connected grid, deformation, metal residue / dirt, contour defects, dents, wire mesh pinholes, protrusions, and pad vias; the inference process adopts a single-stage detection method, which completes target classification and bounding box regression simultaneously in one forward propagation.

7. The method according to claim 1, characterized in that, Step 7 specifically includes: setting the confidence threshold to 0.1, retaining only detection results with a confidence level higher than this threshold; setting the cross-union ratio threshold for non-maximum suppression to 0.5; performing classification statistics according to defect categories, and drawing detection bounding boxes, labeling defect categories and confidence values ​​on the original image.

8. The method according to claim 1, characterized in that, The industrial standard defect determination in step 8 includes: Protrusion defects: The strict standard is that a protrusion height exceeding one-third of the wire grid spacing is considered unqualified, while the lenient standard is that a protrusion height exceeding two-thirds of the wire grid width is considered unqualified. Dirt defects and pinhole defects: The strict standard is that if the defect area exceeds half of the wire mesh end area, it is considered unqualified. The lenient standard is that if the defect area is between half and the full size and the number is more than three, it is considered unqualified. Alternatively, if the defect area exceeds the full size, it is directly considered unqualified.

9. The method according to claim 1, characterized in that, In step 1, the microscopic imaging device includes a UVC protocol-compatible microscope camera with a USB interface and an adjustable light source module; the microscope camera supports multiple resolution settings, with an exposure parameter adjustment range of -12 to 0, a brightness parameter and a contrast parameter adjustment range of 0 to 255, and a frame rate parameter adjustment range of 1 to 120fps.

10. The method according to claim 4, characterized in that, It also includes the step of exporting the trained strain gauge micro-defect detection model into the ONNX standardized format for use in edge computing device deployment; in step 4, the sliding window slice size is set to 640×640 pixels, and the overlapping pixels of adjacent slices are set to 160 pixels.

Images