Probe defect detection method, device, equipment and readable storage medium
By using a preset image matching and connected component nonmaximum suppression algorithm, combined with geometric, grayscale and intensity feature calculations, the problem of poor performance of probe appearance defect detection algorithms is solved, and accurate detection and detailed description of probe groups with different shapes are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- WUHAN ZHONGDAO OPTOELECTRONIC EQUIP CO LTD
- Filing Date
- 2025-12-25
- Publication Date
- 2026-05-12
AI Technical Summary
现有的探针外观缺陷检测算法检测效果不佳，无法准确识别探针组的形态各异导致的缺陷。
The algorithm matches a preset image with the image to be detected, uses a connected component nonmaximum suppression algorithm to extract the image region of the probe group, and identifies defects by calculating geometric features, grayscale features and intensity features. This avoids reliance on neighboring probe groups and gold templates and adapts to situations with different probe group shapes.
It improves the accuracy and reliability of probe defect detection, reduces false negatives and missed positives, and can accurately identify defects within the probe group and provide detailed feature descriptions.
Smart Images
Figure CN121414740B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image recognition technology, and in particular to a probe defect detection method, apparatus, device, and readable storage medium. Background Technology
[0002] A probe card is the interface connecting a testing machine and a semiconductor wafer. The testing machine uses the probe card to inspect the wafer's quality, and the probe is the core component of the probe card; the quality of the probe directly affects the accuracy of wafer quality inspection. Regarding the inspection of probe appearance defects, using an electron microscope offers high precision but low efficiency. Utilizing algorithms to detect appearance defects based on captured probe images can achieve rapid and automated inspection.
[0003] However, current algorithms for detecting defects in probe appearance are not performing well. Summary of the Invention
[0004] This application provides a method, apparatus, device, and readable storage medium for detecting probe defects, aiming to solve the technical problem that current algorithms for detecting probe appearance defects have poor detection effects.
[0005] In a first aspect, embodiments of this application provide a probe defect detection method, the probe defect detection method comprising:
[0006] Acquire an image to be detected, wherein the image to be detected includes at least one probe group, wherein each probe group includes a probe and auxiliary frames adjacent to both sides of the probe;
[0007] The image region of each probe group is extracted from the image to be detected by matching a preset image with the image to be detected. The preset image is a standard image of a probe group.
[0008] For each probe group's image region, the connected component nonmaximum suppression algorithm is used to detect the image region and obtain each detection box;
[0009] For each detection frame, those with an area smaller than the preset standard area but larger than the preset defect area are detected as defects.
[0010] Calculate and output the geometric features, grayscale features, and intensity features of each defect.
[0011] Optionally, the step of using a connected component nonmaximum suppression algorithm to detect the image region for each probe group to obtain each detection box includes:
[0012] For each probe group's image region, multiple detection boxes are detected from the image region;
[0013] According to the preset connectivity rules, multiple connected components are identified based on multiple detection boxes;
[0014] For all bounding boxes in each connected component, select the first bounding box in descending order of confidence, calculate the intersection-union ratio (IU) between the first bounding box and other bounding boxes, and remove other bounding boxes whose IU is greater than the threshold.
[0015] All the detection boxes that are retained after removal are used as the final detection boxes.
[0016] Optionally, before calculating and outputting the geometric features, grayscale features, and intensity features of each defect, the following steps are included:
[0017] For each probe group's image region, for each pixel in the image region, convolution calculation is performed using preset gradient operators in multiple preset directions to obtain the gradient magnitude and gradient direction of each pixel;
[0018] Calculate the average and variance of the gradient magnitude, the consistency of the gradient direction, and the edge curvature for each pixel;
[0019] The edge feature vector of each pixel is constructed based on the average and variance of the gradient magnitude, the consistency of the gradient direction, and the edge curvature.
[0020] Detect the location of pixels whose edge feature vectors exceed a preset range as defects.
[0021] Optionally, before calculating and outputting the geometric features, grayscale features, and intensity features of each defect, the following steps are included:
[0022] For each probe group's image region, a circular region is defined with each pixel in the image region as the center and a preset length as the radius.
[0023] If the difference between the brightness value of a preset number of consecutive pixels within a circular area and the brightness value of the pixel at the center of the circle is greater than a threshold, then the pixel at the center of the circle is determined as the first corner point.
[0024] For each first corner point, calculate the difference in brightness values between the first corner point and a preset number of pixels within a preset neighboring range, and sum the absolute values of the differences in brightness values to obtain the score of the first corner point.
[0025] The image region is divided into grids, and the first corner point with the highest score in each grid is selected as the second corner point.
[0026] If the position of the second corner point is outside the preset position range, the position of the second corner point will be detected as a defect.
[0027] Optionally, before matching the preset image and the image to be detected to extract the image region of each probe group from the image to be detected, the following steps are included:
[0028] The image to be detected is smoothed and denoised.
[0029] Optionally, before matching the preset image and the image to be detected to extract the image region of each probe group from the image to be detected, the following steps are included:
[0030] If the image to be detected is a non-grayscale image, then the image to be detected will be converted to a grayscale image.
[0031] Secondly, embodiments of this application provide a probe defect detection device, the probe defect detection device comprising:
[0032] An acquisition module is used to acquire an image to be detected, wherein the image to be detected includes at least one probe group, wherein each probe group includes a probe and auxiliary frames adjacent to both sides of the probe;
[0033] The matching module is used to match a preset image and an image to be detected, and to extract the image region of each probe group from the image to be detected. The preset image is a standard image of a probe group.
[0034] The detection module is used to detect the image region for each probe group using a connected component nonmaximum suppression algorithm to obtain each detection box;
[0035] The determination module is used to detect defects in each detection frame if the area is smaller than the preset standard area but larger than the preset defect area.
[0036] The calculation module is used to calculate and output the geometric features, grayscale features, and intensity features of each defect.
[0037] Optionally, the detection module is used for:
[0038] For each probe group's image region, multiple detection boxes are detected from the image region;
[0039] According to the preset connectivity rules, multiple connected components are identified based on multiple detection boxes;
[0040] For all bounding boxes in each connected component, select the first bounding box in descending order of confidence, calculate the intersection-union ratio (IU) between the first bounding box and other bounding boxes, and remove other bounding boxes whose IU is greater than the threshold.
[0041] All the detection boxes that are retained after removal are used as the final detection boxes.
[0042] Thirdly, embodiments of this application provide a probe defect detection device, which includes a processor, a memory, and a probe defect detection program stored in the memory and executable by the processor, wherein when the probe defect detection program is executed by the processor, it implements the steps of the probe defect detection method as described above.
[0043] Fourthly, embodiments of this application provide a readable storage medium storing a probe defect detection program, wherein when the probe defect detection program is executed by a processor, it implements the steps of the probe defect detection method as described above.
[0044] The beneficial effects of the technical solutions provided in this application include:
[0045] In this embodiment, an image to be detected is acquired, which includes at least one probe group. Each probe group includes a probe and auxiliary frames adjacent to both sides of the probe. A preset image and the image to be detected are matched to extract the image region of each probe group from the image to be detected. The preset image is a standard image of a probe group. For the image region of each probe group, a connected component nonmaximum suppression algorithm is used to detect the image region to obtain each detection box. For each detection box, detection boxes with an area smaller than a preset standard area but larger than a preset defect area are detected as defects. The geometric features, grayscale features, and intensity features of each defect are calculated and output. In this embodiment, because the probe group is attached to the flexible jelly adhesive, the shape of each probe group in the captured image to be detected is different, which makes the commonly used neighbor unit grayscale comparison algorithm prone to failure, and the detection effect of the gold template grayscale comparison algorithm is also not good. However, the image region of each probe group is extracted based on the preset standard image matching, and then the connected component nonmaximum suppression algorithm is used separately for each probe group. It does not depend on other neighboring probe groups, nor on the gold template, which is more suitable for the situation where the shape of each probe group in the image is different. The connected component nonmaximum suppression algorithm can also effectively avoid redundant detection of defects within each probe group. Compared with the existing detection algorithms, the detection results are more accurate. The standard area of the probe area and the auxiliary frame area on both sides of each probe group is preset and known. Therefore, if the area of a detection box is smaller than the preset standard area and larger than the preset defect area to be concerned, it is said that the detection box is a bad point defect. Further calculation and output of the geometric features, grayscale features and intensity features of the defect are performed to accurately describe the detected defect. Attached Figure Description
[0046] Figure 1 This is a schematic flowchart of an embodiment of the probe defect detection method of this application;
[0047] Figure 2This is a schematic diagram of the image to be detected according to an embodiment of the probe defect detection method of this application;
[0048] Figure 3 This is a partially enlarged schematic diagram of a defect-free probe group according to an embodiment of the probe defect detection method of this application;
[0049] Figure 4 This is a partially enlarged schematic diagram of a defective probe group according to an embodiment of the probe defect detection method of this application.
[0050] Figure 5 This is a functional module diagram of an embodiment of the probe defect detection device of this application;
[0051] Figure 6 This is a schematic diagram of the hardware structure of the probe defect detection device involved in the embodiments of this application. Detailed Implementation
[0052] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.
[0053] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0054] In a first aspect, embodiments of this application provide a method for detecting probe defects.
[0055] In one embodiment, reference is made to Figure 1 , Figure 1 This is a schematic flowchart of an embodiment of the probe defect detection method of this application, as shown below. Figure 1 As shown, the probe defect detection method includes:
[0056] Step S10: Obtain the image to be detected, wherein the image to be detected includes at least one probe group, wherein each probe group includes a probe and auxiliary frames adjacent to both sides of the probe.
[0057] In this embodiment, an image of the probe group is acquired using a high-precision industrial camera to obtain the image to be detected, as referenced. Figure 2 , Figure 2 This is a schematic diagram of the image to be detected according to an embodiment of the probe defect detection method of this application, as shown below. Figure 2 As shown, Figure 2 The middle section contains 20 probe groups from left to right, as shown in the reference. Figure 3 , Figure 3 This is a partially enlarged schematic diagram of a defect-free probe group according to an embodiment of the probe defect detection method of this application, as shown below. Figure 3 As shown, the white part in the middle of a probe assembly is the probe itself, and the adjacent black parts on both sides are probe supports that protect the probe. The probe assembly is typically attached to a flexible gel. Figure 2 The background between multiple probe groups is the flexible jelly glue. Due to slight differences in the attachment position and angle of the probe groups on the flexible jelly glue, the shape of each probe group in the captured image to be detected varies. This makes the commonly used neighboring unit grayscale comparison algorithm prone to failure. The reason it is prone to failure is that the neighboring unit grayscale comparison algorithm needs to compare the differences between the probe group to be detected and the neighboring probe groups. The detection effect of the gold template grayscale comparison algorithm is also not good. The reason for the poor detection effect is that the gold template grayscale comparison algorithm needs to compare the differences between the probe group to be detected and the "perfect probe group".
[0058] Step S20: Match the preset image and the image to be detected, and extract the image region of each probe group from the image to be detected. The preset image is a standard image of a probe group.
[0059] In this embodiment, the preset image can be obtained by acquiring an image of a defect-free probe group under standard conditions. The preset image can be referenced. Figure 3 As shown, the image contains the complete shape of the probe and the auxiliary frames on both sides, serving as a reference for subsequent matching. The matching process can employ a feature-point-based image registration algorithm. First, feature points are extracted from the preset image and the image to be detected. Then, the transformation matrix between the images is calculated using the algorithm. Finally, the transformation matrix is applied to the image to be detected to accurately extract the image region of each probe group. Alternatively, the detection images can be matched from left to right. After matching the image region of the first probe group, the image regions of subsequent probe groups are determined based on the pre-set fixed spacing between each probe group. Through precise matching, the image region of each probe group is accurately extracted, providing high-quality image input data for subsequent defect detection targeting each probe group individually.
[0060] Step S30: For the image region of each probe group, the connected component nonmaximum suppression algorithm is used to detect the image region to obtain each detection box.
[0061] In this embodiment, the main implementation steps are as follows: First, target detection is performed on the image region of each probe group to obtain all target detection boxes; then, the detection boxes are grouped according to a preset connectivity rule to form multiple connected components; next, for all detection boxes in each connected component, the first detection box is selected in descending order of confidence, the intersection-union ratio (IU) between the first detection box and other detection boxes is calculated, and other detection boxes with an IU greater than a threshold are removed; finally, all the detection boxes that remain after removal are taken as the final detected detection boxes. Specifically, refer to the comparison. Figure 3 and Figure 4 ,in, Figure 4 This is a partially enlarged schematic diagram of a defective probe group according to an embodiment of the probe defect detection method of this application. Figure 3 A defect-free probe group, when used to detect image regions using the connected component nonmaximum suppression algorithm, yields three detection boxes: two black boxes and one white box. For Figure 4 The image region of a defective probe group, detected using the connected component nonmaximum suppression algorithm, yields five detection boxes: two black boxes, one white box, and two orange boxes. The core advantage of using the connected component nonmaximum suppression algorithm in this embodiment is that it does not rely on neighboring probe groups or the gold template; instead, it processes each probe group independently, making it ideal for situations where the probe groups in the image to be detected have varying shapes. By using the connected component nonmaximum suppression algorithm, redundant detection of defects within each probe group can be effectively avoided. Compared to existing detection algorithms, it can effectively suppress false detections caused by differences in probe group shapes, preventing normal areas from being misclassified as defects, resulting in more accurate detection results. It should be noted that the connected component nonmaximum suppression algorithm is used independently for the image region of each probe group, making it particularly suitable for the entire surface of the probe group (i.e., the entire surface of the probe group). Figure 3 Defect detection of small-area defects (shown as the overall area of 2 black boxes and 1 white box).
[0062] Step S40: For each detection frame, detect frames with an area smaller than the preset standard area but larger than the preset defect area as defects.
[0063] In this embodiment, each probe group includes a probe region (e.g., Figure 3 (as shown in the white box in the image) and the auxiliary frame areas on both sides (as shown in the image) Figure 3The standard area (shown by the two black boxes in the image) is a preset known area, which can be obtained by measuring in a defect-free standard image. The preset defect area is determined based on actual production needs and the severity of the defect, and is usually the area of the smallest detectable defect. Detection frames with an area smaller than the preset standard area but larger than the preset defect area are identified as defects. The principle is that the area of the normal probe region should be close to the standard area, while the defect area is usually a small area anomaly, but it cannot be too small (such as noise points). Therefore, setting an area threshold range can effectively distinguish between defects and noise. When the area of the detection frame is smaller than the standard area but larger than the preset defect area, it indicates that there is an anomaly in that area, which may be a scratch, stain, or small dent on the probe surface. Continue to refer to... Figure 4 ,for Figure 4 The two orange boxes in the detection algorithm have areas significantly smaller than the areas of the preset standard black and white boxes. If the area of these two orange boxes is larger than the preset defect area, then the defect is detected. By setting a reasonable area threshold, false negatives and false positives can be reduced, improving the accuracy and reliability of detection.
[0064] Step S50: Calculate and output the geometric features, grayscale features and intensity features of each defect.
[0065] In this embodiment, for each detected defect, its geometric features, grayscale features, and intensity features are calculated to comprehensively describe the characteristics of the defect. Geometric features include the area, perimeter, location, circumscribed rectangle, aspect ratio, and number of holes of the defect region. These features describe the shape, size, and spatial distribution of the defect. Grayscale features include average grayscale, grayscale standard deviation, grayscale variation range, and contrast. These features reflect the brightness information and internal uniformity of the defect region. Intensity features include the degree of difference between the defect region and the surrounding background, used to assess the severity of the defect. By calculating these features, the detected defects can be accurately described, providing data support for subsequent defect classification and analysis. For example, a high grayscale standard deviation usually indicates large brightness variations within the defect region, possibly due to surface unevenness or stains; while a specific aspect ratio may indicate the type of defect (such as a linear defect or a circular defect). Through comprehensive analysis of these features, the detection system can not only detect defects but also classify them, providing richer information for quality control.
[0066] In this embodiment, a high-precision industrial camera is used to acquire images of the probe group to obtain the image to be inspected. The preset image can be obtained by acquiring an image of a defect-free probe group under standard conditions. By accurately matching the preset image and the image to be inspected, the image region of each probe group is accurately extracted, providing high-quality image input data for subsequent defect detection of each probe group individually. The core advantage of using the connected component nonmaximum suppression algorithm is that it does not rely on other neighboring probe groups or the golden template, but processes each probe group individually, making it very suitable for situations where the shapes of each probe group in the image to be inspected vary. By using the connected component nonmaximum suppression algorithm, redundant detection of defects within each probe group can be effectively avoided. Compared with existing detection algorithms, it can effectively suppress false detections caused by differences in probe group shapes, resulting in more accurate detection results. This algorithm is particularly suitable for probe defect detection because probe defects usually manifest as small abnormal areas, and this algorithm can accurately identify these small defects without misclassifying normal areas as defects. By reasonably setting the area threshold, false negatives and false positives can be reduced, improving the accuracy and reliability of detection. For each detected defect, its geometric features, grayscale features, and intensity features are calculated to comprehensively describe the characteristics of the defect. By calculating these features, detected defects can be accurately described, providing data support for subsequent defect classification and analysis. Through comprehensive analysis of these features, the detection system can not only identify defects but also classify them, providing richer information for quality control.
[0067] Further, in one embodiment, step S30 includes:
[0068] For each probe group's image region, multiple detection boxes are detected from the image region;
[0069] According to the preset connectivity rules, multiple connected components are identified based on multiple detection boxes;
[0070] For all bounding boxes in each connected component, select the first bounding box in descending order of confidence, calculate the intersection-union ratio (IU) between the first bounding box and other bounding boxes, and remove other bounding boxes whose IU is greater than the threshold.
[0071] All the detection boxes that are retained after removal are used as the final detection boxes.
[0072] In this embodiment, the Connected Component Non-Maximum Suppression (CCNMSA) algorithm avoids the limitations of the traditional NMS algorithm in probe defect detection by using a grouping approach. The traditional NMS algorithm performs non-maximum suppression globally, easily misclassifying defects at different locations within the same probe group as the same target and suppressing them. CCNMSA, however, first groups detection boxes that may belong to the same target according to a preset connectivity rule, forming connected components. Then, it applies NMS independently within each connected component. This ensures that multiple defects within the same probe group are preserved without redundant detection. This algorithm significantly improves detection accuracy, especially when multiple small defects exist within a probe group, effectively avoiding missed detections. Furthermore, by setting a reasonable cross-union threshold, it avoids redundant detection caused by overlapping detection boxes.
[0073] Further, in one embodiment, before step S50, the following steps are included:
[0074] For each probe group's image region, for each pixel in the image region, convolution calculation is performed using preset gradient operators in multiple preset directions to obtain the gradient magnitude and gradient direction of each pixel;
[0075] Calculate the average and variance of the gradient magnitude, the consistency of the gradient direction, and the edge curvature for each pixel;
[0076] The edge feature vector of each pixel is constructed based on the average and variance of the gradient magnitude, the consistency of the gradient direction, and the edge curvature.
[0077] Detect the location of pixels whose edge feature vectors exceed a preset range as defects.
[0078] In this embodiment, the Multi-directional Gradient Detection Algorithm (MDGA) effectively overcomes the impact of probe edge shape differences on defect detection by calculating gradients in multiple directions. Traditional single-directional gradient detection algorithms are prone to failure when the probe edge shape is irregular, while MDGA, by calculating gradients in multiple directions, can capture edge features more comprehensively. Specifically, the algorithm uses preset gradient operators in multiple directions (such as the Sobel operator in 0°, 45°, 90°, 135°, etc.) to convolve each pixel in the image to obtain the gradient value in each direction. Then, the average and variance of the gradient magnitude, the consistency of the gradient direction, and the edge curvature of each pixel are calculated. These features together constitute the edge feature vector of the pixel. The edge feature vector of a normal probe has a specific distribution pattern. When the edge feature vector exceeds the preset range, it indicates that there is an anomaly at the pixel location, i.e., a defect. The beneficial effect of this algorithm is that it can effectively detect subtle changes in the probe edge, such as edge breaks caused by probe bending or surface unevenness, significantly improving the detection rate of edge defects. It should be noted that, while both steps target the image region of each probe group, this differs from the approach taken by the connected component nonmaximum suppression algorithm in step S30, which targets the entire surface of each probe group (i.e., the entire surface of the probe group). Figure 3 For small-area defect detection (shown as the entire area of two black boxes and one white box), the multi-directional gradient detection algorithm focuses on detecting edge defects (i.e., defects in the entire area of the two black boxes and one white box) of each probe group. Figure 3 The method of detecting edge defects (such as breaks or bends) at the junction of white and black areas is combined with the defect detection of small-area bad spots on the overall surface of the probe group by the connected component nonmaximum suppression algorithm in step S30. This allows for comprehensive detection of defects in different parts and of different types of the probe group. Furthermore, in conjunction with step S50, the method calculates and outputs the geometric features, grayscale features, and intensity features of each defect in different parts and of different types of the probe group. This results in comprehensive and mutually verifiable defect information, enabling more accurate detection of appearance defects in the probe group.
[0079] Further, in one embodiment, before step S50, the following steps are included:
[0080] For each probe group's image region, a circular region is defined with each pixel in the image region as the center and a preset length as the radius.
[0081] If the difference between the brightness value of a preset number of consecutive pixels within a circular area and the brightness value of the pixel at the center of the circle is greater than a threshold, then the pixel at the center of the circle is determined as the first corner point.
[0082] For each first corner point, calculate the difference in brightness values between the first corner point and a preset number of pixels within a preset neighboring range, and sum the absolute values of the differences in brightness values to obtain the score of the first corner point.
[0083] The image region is divided into grids, and the first corner point with the highest score in each grid is selected as the second corner point.
[0084] If the position of the second corner point is outside the preset position range, the position of the second corner point will be detected as a defect.
[0085] In this embodiment, the FastNMS (Fast Non-Maximum Suppression) algorithm is used to quickly detect corners and identify morphological differences on the probe surface. The core idea of the FastNMS algorithm is to quickly determine whether a point is a corner by comparing the grayscale / brightness values of pixels on a ring with those of the center point. A corner is a point in an image where a certain attribute is particularly prominent. The specific steps are as follows: First, a circular region is defined with each pixel as the center and a preset length as the radius. Then, it is checked whether there are a preset number of consecutive pixels within the circular region whose brightness value differs from the brightness value of the pixel at the center position by a preset threshold. If so, this pixel is identified as the first corner. Next, the score of each first corner is calculated, which is the sum of the absolute values of the brightness differences between the corner and its neighboring pixels. Then, the image region is divided into grids, and the first corner with the highest score within each grid is selected as the second corner. This performs non-maximum suppression within a local area, selecting the most prominent corner to avoid redundant detection. Finally, if the position of the second corner exceeds a preset position range (e.g., it appears in a certain location), the algorithm will determine the corner's position. Figure 4 If the area within the black or white box shown is a defect, then the location is determined to be a pit or bump defect. The advantage of this algorithm is its ability to quickly detect morphological differences such as pits and bumps on the probe surface, making it particularly suitable for detecting minute depressions or protrusions. It should be noted that this algorithm also targets the image region of each probe group, which differs from the connected component non-maximum suppression algorithm in step S30, which targets the entire surface of each probe group (i.e., the entire surface of the probe group). Figure 3 The algorithm detects small-area defects (the entire area shown by two black boxes and one white box), and focuses on detecting edge defects (i.e., defects in the edge area of each probe group) using a multi-directional gradient detection algorithm. Figure 3 (Whether there are edge defects such as breaks or bends at the edge where the white and black meet) The non-maximum suppression corner detection algorithm focuses on detecting bumps and pits in each probe group. Combined with the first two detection methods, it can make more comprehensive detection of different parts and different types of defects in the probe group. In addition, combined with step S50, it calculates and outputs the geometric features, grayscale features and intensity features of each defect in different parts and different types of the probe group, so as to form comprehensive and cross-verifiable defect information, thereby enabling more accurate detection of appearance defects in the probe group.
[0086] Further, in one embodiment, before step S20, the following steps are included:
[0087] The image to be detected is smoothed and denoised.
[0088] In this embodiment, the image to be detected is smoothed and denoised, primarily to eliminate noise in the image and improve the accuracy of subsequent matching and detection. A suitable smoothing and denoising algorithm can be selected based on the type of image noise. Commonly used smoothing and denoising algorithms include Gaussian filtering and median filtering. Gaussian filtering smooths the image using a weighted average, effectively removing Gaussian noise while preserving the main features of the image. Median filtering, on the other hand, eliminates salt-and-pepper noise by taking the median value of the pixel's neighborhood.
[0089] Further, in one embodiment, before step S20, the following steps are included:
[0090] If the image to be detected is a non-grayscale image, then the image to be detected will be converted to a grayscale image.
[0091] In this embodiment, converting non-grayscale images (such as RGB color images) to grayscale images simplifies the image processing workflow and improves computational efficiency. Color images contain pixel values in three channels, while grayscale images have only one channel, resulting in less computation and faster processing. In probe defect detection, the defect characteristics of the probe are mainly reflected in grayscale changes, while color information has a relatively small impact on defect detection. Therefore, converting color images to grayscale images not only preserves important information but also reduces computational complexity and improves detection speed. The specific conversion method uses a weighted average method: Gray = 0.299R + 0.587G + 0.114B, where R, G, and B are the pixel values of the red, green, and blue channels, respectively. This conversion method effectively preserves the image's brightness information while simplifying the image processing workflow. The beneficial effects of converting non-grayscale images to grayscale images are a significant reduction in the algorithm's computational complexity, improved detection speed, and ensured detection accuracy.
[0092] Secondly, embodiments of this application also provide a probe defect detection device.
[0093] In one embodiment, reference is made to Figure 5 , Figure 5 This is a functional module diagram of an embodiment of the probe defect detection device of this application, as shown below. Figure 5 As shown, the probe defect detection device includes:
[0094] The acquisition module 10 is used to acquire an image to be detected, wherein the image to be detected includes at least one probe group, wherein each probe group includes a probe and auxiliary frames adjacent to both sides of the probe.
[0095] Matching module 20 is used to match a preset image and an image to be detected, and to extract the image region of each probe group from the image to be detected. The preset image is a standard image of a probe group.
[0096] The detection module 30 is used to detect the image region for each probe group using a connected component nonmaximum suppression algorithm to obtain each detection box;
[0097] The determination module 40 is used to detect each detection frame as a defect if the area of the detection frame is smaller than the preset standard area but larger than the preset defect area.
[0098] The calculation module 50 is used to calculate and output the geometric features, grayscale features and intensity features of each defect.
[0099] Furthermore, in one embodiment, the detection module 30 is used for:
[0100] For each probe group's image region, multiple detection boxes are detected from the image region;
[0101] According to the preset connectivity rules, multiple connected components are identified based on multiple detection boxes;
[0102] For all bounding boxes in each connected component, select the first bounding box in descending order of confidence, calculate the intersection-union ratio (IU) between the first bounding box and other bounding boxes, and remove other bounding boxes whose IU is greater than the threshold.
[0103] All the detection boxes that are retained after removal are used as the final detection boxes.
[0104] Furthermore, in one embodiment, the probe defect detection device further includes an edge defect detection module, used for:
[0105] For each probe group's image region, for each pixel in the image region, convolution calculation is performed using preset gradient operators in multiple preset directions to obtain the gradient magnitude and gradient direction of each pixel;
[0106] Calculate the average and variance of the gradient magnitude, the consistency of the gradient direction, and the edge curvature for each pixel;
[0107] The edge feature vector of each pixel is constructed based on the average and variance of the gradient magnitude, the consistency of the gradient direction, and the edge curvature.
[0108] Detect the location of pixels whose edge feature vectors exceed a preset range as defects.
[0109] Furthermore, in one embodiment, the probe defect detection device further includes a corner defect detection module, used for:
[0110] For each probe group's image region, a circular region is defined with each pixel in the image region as the center and a preset length as the radius.
[0111] If the difference between the brightness value of a preset number of consecutive pixels within a circular area and the brightness value of the pixel at the center of the circle is greater than a threshold, then the pixel at the center of the circle is determined as the first corner point.
[0112] For each first corner point, calculate the difference in brightness values between the first corner point and a preset number of pixels within a preset neighboring range, and sum the absolute values of the differences in brightness values to obtain the score of the first corner point.
[0113] The image region is divided into grids, and the first corner point with the highest score in each grid is selected as the second corner point.
[0114] If the position of the second corner point is outside the preset position range, the position of the second corner point will be detected as a defect.
[0115] Furthermore, in one embodiment, the probe defect detection device further includes a smoothing and noise reduction processing module, used for:
[0116] The image to be detected is smoothed and denoised.
[0117] Furthermore, in one embodiment, the probe defect detection device further includes a grayscale conversion module, used for:
[0118] If the image to be detected is a non-grayscale image, then the image to be detected will be converted to a grayscale image.
[0119] The functions of each module in the probe defect detection device correspond to the steps in the probe defect detection method embodiment, and their functions and implementation processes will not be described in detail here.
[0120] Thirdly, embodiments of this application provide a probe defect detection device.
[0121] Reference Figure 6 , Figure 6 This is a schematic diagram of the hardware structure of the probe defect detection device involved in the embodiments of this application. In the embodiments of this application, the probe defect detection device may include a processor, a memory, a communication interface, and a communication bus.
[0122] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.
[0123] Communication interfaces include input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting internal components of the probe defect detection equipment, as well as interfaces used for interconnecting the probe defect detection equipment with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.
[0124] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.
[0125] The processor can be a general-purpose processor, which can call the probe defect detection program stored in the memory and execute the probe defect detection method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the probe defect detection program is called can be referred to in the various embodiments of the probe defect detection method of this application, and will not be repeated here.
[0126] Those skilled in the art will understand that Figure 6 The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0127] Fourthly, embodiments of this application also provide a readable storage medium.
[0128] The present application has a readable storage medium storing a probe defect detection program, wherein when the probe defect detection program is executed by a processor, it implements the steps of the probe defect detection method described above.
[0129] The method implemented when the probe defect detection program is executed can be referred to in various embodiments of the probe defect detection method of this application, and will not be repeated here.
[0130] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0131] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.
[0132] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.
[0133] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.
[0134] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.
[0135] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.
[0136] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. A method for detecting probe defects, characterized in that, The probe defect detection method includes: Acquire an image to be detected, wherein the image to be detected includes at least one probe group, wherein each probe group includes a probe and auxiliary frames adjacent to both sides of the probe; The image region of each probe group is extracted from the image to be detected by matching a preset image with the image to be detected. The preset image is a standard image of a probe group. For each probe group's image region, a connected component nonmaximum suppression algorithm is used to detect the image region and obtain each detection box; For each detection frame, those with an area smaller than the preset standard area but larger than the preset defect area are detected as defects. Calculate and output the geometric features, grayscale features, and intensity features of each defect; For each probe group's image region, the connected component nonmaximum suppression algorithm is used to detect the image region, resulting in each detection box, including: For each probe group's image region, multiple detection boxes are detected from the image region; According to the preset connectivity rules, multiple connected components are identified based on multiple detection boxes; For all bounding boxes in each connected component, select the first bounding box in descending order of confidence, calculate the intersection-union ratio (IU) between the first bounding box and other bounding boxes, and remove other bounding boxes whose IU is greater than the threshold. All the detection boxes that are retained after removal are used as the final detection boxes.
2. The probe defect detection method as described in claim 1, characterized in that, Before calculating and outputting the geometric features, grayscale features, and intensity features of each defect, the following steps are included: For each probe group's image region, for each pixel in the image region, convolution calculation is performed using preset gradient operators in multiple preset directions to obtain the gradient magnitude and gradient direction of each pixel; Calculate the average and variance of the gradient magnitude, the consistency of the gradient direction, and the edge curvature for each pixel; The edge feature vector of each pixel is constructed based on the average and variance of the gradient magnitude, the consistency of the gradient direction, and the edge curvature. Detect the location of pixels whose edge feature vectors exceed a preset range as defects.
3. The probe defect detection method as described in claim 1, characterized in that, Before calculating and outputting the geometric features, grayscale features, and intensity features of each defect, the following steps are included: For each probe group's image region, a circular region is defined with each pixel in the image region as the center and a preset length as the radius. If the difference between the brightness value of a preset number of consecutive pixels within a circular area and the brightness value of the pixel at the center of the circle is greater than a threshold, then the pixel at the center of the circle is determined as the first corner point. For each first corner point, calculate the difference in brightness values between the first corner point and a preset number of pixels within a preset neighboring range, and sum the absolute values of the differences in brightness values to obtain the score of the first corner point. The image region is divided into grids, and the first corner point with the highest score in each grid is selected as the second corner point. If the position of the second corner point is outside the preset position range, the position of the second corner point will be detected as a defect.
4. The probe defect detection method as described in claim 1, characterized in that, Before matching the preset image and the image to be detected, and extracting the image region of each probe group from the image to be detected, the process includes: The image to be detected is smoothed and denoised.
5. The probe defect detection method as described in claim 1, characterized in that, Before matching the preset image and the image to be detected, and extracting the image region of each probe group from the image to be detected, the process includes: If the image to be detected is a non-grayscale image, then the image to be detected will be converted to a grayscale image.
6. A probe defect detection device, characterized in that, The probe defect detection device includes: An acquisition module is used to acquire an image to be detected, wherein the image to be detected includes at least one probe group, wherein each probe group includes a probe and auxiliary frames adjacent to both sides of the probe; The matching module is used to match a preset image and an image to be detected, and to extract the image region of each probe group from the image to be detected. The preset image is a standard image of a probe group. The detection module is used to detect the image region for each probe group using a connected component nonmaximum suppression algorithm to obtain each detection box; The determination module is used to detect defects in each detection frame if the area is smaller than the preset standard area but larger than the preset defect area. The calculation module is used to calculate and output the geometric features, grayscale features, and intensity features of each defect; The detection module is used for: For each probe group's image region, multiple detection boxes are detected from the image region; According to the preset connectivity rules, multiple connected components are identified based on multiple detection boxes; For all bounding boxes in each connected component, select the first bounding box in descending order of confidence, calculate the intersection-union ratio (IU) between the first bounding box and other bounding boxes, and remove other bounding boxes whose IU is greater than the threshold. All the detection boxes that are retained after removal are used as the final detection boxes.
7. A probe defect detection device, characterized in that, The probe defect detection device includes a processor, a memory, and a probe defect detection program stored in the memory and executable by the processor, wherein when the probe defect detection program is executed by the processor, it implements the steps of the probe defect detection method as described in any one of claims 1 to 5.
8. A readable storage medium, characterized in that, The readable storage medium stores a probe defect detection program, wherein when the probe defect detection program is executed by a processor, it implements the steps of the probe defect detection method as described in any one of claims 1 to 5.