Methods, systems, equipment and media for flexible detection of hole and groove defects in finished sheet metal.

The flexible detection method for hole and groove defects in finished sheet metal, which combines cropping, image preprocessing, and affine matrix transformation, solves the problems of low efficiency, insufficient accuracy, and susceptibility to interference in traditional detection methods, and achieves efficient and accurate hole and groove identification and defect judgment.

CN123118577APending Publication Date: 2026-11-17GUANGZHOU SHENLU AUTOMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611404936.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-10
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing technologies for identifying holes and grooves in finished sheet metal are insufficient to meet actual industrial needs in terms of accuracy, efficiency, and robustness. Traditional manual inspection is inefficient, costly, and lacks precision, while machine vision systems are susceptible to interference from external factors, leading to unstable recognition results.

Method used

A flexible detection method for hole and groove defects in finished sheet metal is adopted. By cutting candidate areas, image preprocessing, edge contour processing, drawing screening and affine matrix transformation, combined with tolerance parameters, the measured hole and groove are matched with the designed hole and groove. The method automatically completes drawing screening, posture traversal and multi-region recognition, and eliminates imaging distortion and position deviation.

Benefits of technology

It improves the accuracy and efficiency of detection, reduces reliance on manual labor, comprehensively reflects processing quality, and outputs target defect information to provide data support for equipment parameter adjustment and process improvement, solving the problem that existing technologies cannot meet industrial needs in terms of identification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118577A_ABST
    Figure CN123118577A_ABST
Patent Text Reader

Abstract

This invention provides a flexible detection method, system, equipment, and medium for hole and groove defects in finished sheet metal, belonging to the field of quality inspection technology. The method includes: cropping candidate regions from a sheet metal image; obtaining edge images through preprocessing and edge contour processing; and selecting measured hole areas, chipping candidate areas, and groove candidate areas from the candidate images based on the edge images. Valid drawings are then screened using both front and back views, and tolerance parameters are determined according to the sheet type. An affine matrix is ​​established from the candidate images to the valid drawings. Geometric analysis is performed on the measured hole areas and groove candidate areas, and the target geometric features in the drawing coordinate system are obtained through affine matrix transformation. A pose traversal is performed by combining the valid drawings and target geometric features, and hole and groove designs are matched according to the tolerance parameters to obtain target matching results. Finally, the target matching results and chipping candidate areas are comprehensively analyzed and defect judgment is performed, outputting the target defect information of the target sheet metal.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of quality inspection technology, and in particular to a flexible detection method, system, equipment and medium for hole and groove defects in finished sheet metal. Background Technology

[0002] To ensure the quality of finished sheet materials, inspection is required after production. However, current slot and hole recognition technology still has significant shortcomings. Traditional manual inspection methods are inefficient, costly, lack accuracy, and are highly subjective, making them unsuitable for high-speed assembly line operations and unable to guarantee consistent quality judgment. While subsequent machine vision systems have provided some improvement, they are still susceptible to interference from external factors such as complex surface textures, stains, and changes in workshop lighting, leading to unstable recognition results. Ultimately, slot and hole recognition fails to meet actual industrial needs in terms of accuracy, efficiency, and robustness. Summary of the Invention

[0003] The main objective of this invention is to provide a flexible detection method, system, equipment, and medium for slot and hole defects in finished sheet metal, aiming to solve the problem that the accuracy, efficiency, and robustness of slot and hole identification in the prior art are insufficient to meet the actual industrial needs.

[0004] In a first aspect, embodiments of the present invention provide a method for flexible detection of hole and groove defects in finished sheet metal, comprising: Candidate images are obtained by cropping the candidate region based on the effective range of the target board in the board image; The candidate image is preprocessed and edge contour processing is performed to obtain the corresponding edge image. The candidate image is then filtered and analyzed based on the edge image to obtain the corresponding target candidate region in the candidate image. The target candidate region includes the measured hole region, the chipping candidate region, and the slot candidate region. By using both the front and back sides to attempt to filter the images of the board material to obtain the corresponding valid drawings, and by determining the corresponding tolerance parameters according to the board type corresponding to the target board material; Determine the drawing coordinate system corresponding to the valid drawing and the image coordinate system corresponding to the candidate image, and establish an affine matrix from the candidate image to the valid drawing based on the drawing coordinate system and the image coordinate system; Geometric analysis is performed on the measured hole area and the slot candidate area to obtain initial geometric features, and the initial geometric features are transformed according to the affine matrix to obtain the target geometric features corresponding to the drawing coordinate system. The attitude traversal is performed based on the valid drawings and the target geometric features, and the measured holes and slots are matched with the designed holes and slots in the valid drawings to obtain the target matching result by combining the tolerance parameters. Based on the target matching results and the defect candidate regions, result analysis and defect determination are performed to obtain the target defect information corresponding to the target board.

[0005] Secondly, embodiments of the present invention provide a flexible detection system for hole and groove defects in finished sheet metal, comprising: The region cropping module is used to crop the candidate region based on the effective range of the target board in the board image to obtain the candidate image; The image processing module is used to preprocess and process the edge contour of the candidate image to obtain the corresponding edge image, and to filter and analyze the candidate image based on the edge image to obtain the corresponding target candidate region in the candidate image. The target candidate region includes the measured hole region, the chipping candidate region, and the slot candidate region. The drawing filtering module is used to attempt to filter the board image using both the front and back sides to obtain the corresponding valid drawings, and to determine the corresponding tolerance parameters according to the board type corresponding to the target board. The coordinate transformation module is used to determine the drawing coordinate system corresponding to the valid drawing and the image coordinate system corresponding to the candidate image, and to establish an affine matrix from the candidate image to the valid drawing based on the drawing coordinate system and the image coordinate system. The feature processing module is used to perform geometric analysis on the measured hole area and the slot candidate area to obtain initial geometric features, and to perform information transformation on the initial geometric features according to the affine matrix to obtain the target geometric features corresponding to the drawing coordinate system. The information matching module is used to perform attitude traversal based on the valid drawings and the target geometric features, and combine the tolerance parameters to match the measured holes and slots with the designed holes and slots in the valid drawings to obtain the target matching result; The defect determination module is used to perform result analysis and defect determination based on the target matching result and the chipping candidate area to obtain the target defect information corresponding to the target board.

[0006] Thirdly, embodiments of the present invention also provide a terminal device, the terminal device including a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing communication between the processor and the memory, wherein when the computer program is executed by the processor, it implements the steps of any of the flexible detection methods for hole and groove defects in finished sheet metal as provided in the present invention specification.

[0007] Fourthly, embodiments of the present invention also provide a storage medium for computer-readable storage, characterized in that the storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of any of the flexible detection methods for hole and groove defects in finished sheet metal provided in this specification.

[0008] This invention provides a method, system, equipment, and medium for flexible detection of hole and groove defects in finished sheet metal. The method includes: cropping a candidate region based on the effective range of the target sheet metal in a sheet metal image to obtain a candidate image; preprocessing and edge contour processing of the candidate image to obtain a corresponding edge image; and filtering and analyzing the candidate image based on the edge image to obtain a corresponding target candidate region in the candidate image, the target candidate region including the measured hole region, the chipping candidate region, and the groove candidate region; using both front and back views to perform drawing filtering on the sheet metal image to obtain a corresponding valid drawing, and determining the corresponding tolerance parameter based on the sheet metal type corresponding to the target sheet metal; and determining the valid drawing. The system establishes an affine matrix between the candidate image and the valid drawing based on the drawing coordinate system and the image coordinate system. Geometric analysis is performed on the measured hole area and the candidate slot area to obtain initial geometric features. Information transformation is then performed on these initial geometric features using the affine matrix to obtain the corresponding target geometric features in the drawing coordinate system. An orientation traversal is performed based on the valid drawing and the target geometric features, and tolerance parameters are used to match the measured holes and slots with the designed holes and slots in the valid drawing to obtain the target matching result. Based on the target matching result and the defect candidate area, result analysis and defect judgment are performed to obtain the target defect information corresponding to the target board material. This scheme utilizes an affine matrix to align the image and drawing coordinate systems and sets tolerance parameters according to the board type, effectively eliminating imaging distortion and positional deviation, minimizing the matching error between the measured holes and slots and the design drawing, thereby avoiding false detections and missed detections that are prone to occur during manual comparison. Simultaneously, it automatically completes drawing screening, orientation traversal, and multi-region recognition, eliminating manual calibration, significantly shortening the inspection cycle, reducing reliance on the quality inspector's experience, and improving the overall efficiency of the production line. Furthermore, in terms of detection dimensions, it not only covers hole geometry and slot morphology but also integrates chipping edge defect analysis, comprehensively evaluating shape and position deviations and surface damage to fully reflect the true processing quality and eliminate blind spots in detection. The final output of corresponding target defect information provides data support for subsequent equipment parameter tuning and process improvement. This method also solves the problem that existing technologies for slot and hole identification struggle to meet practical industrial needs in terms of accuracy, efficiency, and robustness. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A flowchart illustrating a flexible detection method for hole and groove defects in finished sheet metal provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a process for obtaining target geometric features according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a scenario for the flexible detection method for hole and groove defects in finished sheet metal provided in this embodiment; Figure 4 This is a schematic diagram of the module structure of a flexible detection system for hole and groove defects in finished sheet metal provided in an embodiment of the present invention; Figure 5 This is a schematic block diagram of a terminal device provided in an embodiment of the present invention. Detailed Implementation

[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0012] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0013] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0014] This invention provides a method, system, equipment, and medium for flexible detection of hole and groove defects in finished sheet metal. The method for flexible detection of hole and groove defects in finished sheet metal can be applied to terminal devices, such as tablet computers, laptops, desktop computers, personal digital assistants, and wearable devices. The terminal device can be a server or a server cluster.

[0015] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0016] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a flexible detection method for hole and groove defects in finished sheet metal provided in an embodiment of the present invention.

[0017] like Figure 1 As shown, the flexible detection method for hole and groove defects in the finished sheet includes steps S101 to S107.

[0018] Step S101: Crop the candidate region according to the effective range of the target board in the board image to obtain the candidate image.

[0019] For example, an image acquisition device is used to acquire images of the finished board material, i.e. the target board material, to obtain the corresponding board material image. Then, the candidate area is cropped according to the effective range of the target board material in the board material image, thereby removing invalid areas such as the outside of the board material and the film in the board material image, so as to obtain the candidate image, i.e., the image area related to the target board material.

[0020] Step S102: Preprocess the candidate image and perform edge contour processing to obtain the corresponding edge image, and filter and analyze the candidate image according to the edge image to obtain the corresponding target candidate region in the candidate image. The target candidate region includes the measured hole region, the chipping candidate region and the slot candidate region.

[0021] For example, morphological processing is used to preprocess the candidate image, such as denoising and filling internal holes, and edge contour enhancement processing is applied to the preprocessed candidate image to obtain the corresponding edge image.

[0022] For example, candidate regions in the edge image are screened based on indicators such as area, shape, and maximum effective borehole size, and regions that do not meet the requirements are eliminated, thereby forming effective measured hole regions and initial slot candidate regions. At the same time, based on the location information of each region, regions located at the edge of the image are marked as defect candidate regions and retained as auxiliary detection information for use in subsequent leak re-judgment.

[0023] For example, the initial slot candidate region is subjected to connected component splitting, and L-shaped and other abnormal regions are filtered out according to the geometric features of the slot. Then, the region is scaled and transformed according to the image scale required for slot matching, and finally the standardized slot candidate region is obtained.

[0024] For example, the measured hole area, the defect candidate area, and the slot candidate area together constitute the target candidate area.

[0025] In some embodiments, the step of preprocessing and edge contour processing of the candidate image to obtain the corresponding edge image includes: denoising the candidate image using a bilateral filter combining spatial proximity weights and color proximity weights to obtain a denoised image; setting a segmentation threshold and determining the effective brightness range corresponding to the denoised image based on the segmentation threshold; determining target segmentation points based on the effective brightness range and dividing the histogram corresponding to the denoised image into a left sub-histogram and a right sub-histogram based on the target segmentation points; dynamically adjusting the first brightness mapping range corresponding to the first sub-region of the left sub-histogram and the second brightness mapping range corresponding to the second sub-region of the right sub-histogram based on the distribution weights of the left sub-histogram and the right sub-histogram; adjusting the brightness of the denoised image based on the first brightness mapping range and the second brightness mapping range to obtain a brightness-adjusted brightness image; dynamically adjusting the exponential factor of the gamma correction algorithm based on the average pixel value of the brightness image as a control benchmark, and performing contrast equalization processing on the brightness image based on the exponential factor and the gamma correction algorithm to obtain a balanced image; and performing image edge processing on the balanced image to obtain the edge image corresponding to the candidate image.

[0026] For example, a neighborhood is defined around each pixel in the candidate image. For each neighboring pixel in the neighborhood, the spatial distance between the neighboring pixel and the current pixel is calculated to determine the spatial proximity weight. Then, the difference between the gray value of the neighboring pixel and the gray value of the current pixel is calculated, and the color proximity weight is determined based on the difference. The new gray value of the current pixel is obtained by weighted averaging of the gray values ​​of all pixels in the neighborhood according to the combined result of these two weights, thus obtaining the denoised image.

[0027] For example, a segmentation threshold is set based on expert experience or historical data. This threshold indicates the minimum number of pixels a gray level must contain to be considered "valid". Starting from the region with the lowest gray value, each gray level is checked upwards until the first gray level with a pixel count equal to or exceeding the segmentation threshold is found, which serves as the lower limit of the effective brightness range. Similarly, starting from the region with the highest gray value, each gray level is checked downwards until the first gray level with a pixel count equal to or exceeding the segmentation threshold is found, which serves as the upper limit of the effective brightness range. Thus, the effective brightness range is determined based on both the upper and lower limits of the effective brightness range.

[0028] For example, within the effective brightness range, each gray level is multiplied by the number of pixels at that gray level, all products are summed, and then divided by the total number of pixels to obtain a weighted average brightness value. The location of this average brightness value is the target segmentation point.

[0029] For example, using the target segmentation point as the boundary, the histogram corresponding to the denoised image is... Figure 1 It is divided into two parts: the left sub-histogram is formed by the target segmentation point to the left, and the right sub-histogram is formed by the target segmentation point to the right.

[0030] For example, the brightness mapping range of the left sub-histogram is determined by the proportion of its pixel count to the total number of pixels globally; the larger the proportion, the wider the first brightness mapping range allocated. Similarly, the right sub-histogram is allocated a second brightness mapping range according to its proportion. This dynamic allocation mechanism ensures that each sub-region receives an expansion space commensurate with its importance, avoiding the problem of "the strong getting stronger and the weak getting weaker" in traditional equalization.

[0031] For example, for each pixel in the denoised image, it is first determined whether it belongs to the left or right sub-histogram. Then, based on the brightness mapping range of its respective sub-histogram, the pixel's original grayscale value is mapped to a new brightness value, thereby obtaining a brightness-adjusted image. After this adjustment, pixels originally concentrated in low-brightness areas are stretched to a wider brightness range, the overall brightness distribution of the image is more uniform, details in dark areas are revealed, and the visual effect is significantly improved.

[0032] For example, the average brightness value of the image is statistically analyzed. Then, based on the relationship between the original brightness value of each pixel and the average brightness value, and using the average pixel value of the image as the adjustment benchmark, an exponential factor is dynamically calculated for each pixel location. For pixels that are too dark, the exponential factor is adjusted to brighten them; for pixels that are too bright, the exponential factor is adjusted to darken them. This adaptive adjustment can effectively compress the brightness difference between dark and bright areas in the image, making the transition between light and dark more natural and the overall contrast more balanced, thereby obtaining a balanced image.

[0033] For example, edge detection processing is performed on the enhanced equalized image to extract edge contour information in the image, and finally the edge image corresponding to the candidate image is output for subsequent applications such as hole and slot detection.

[0034] In some implementations, the step of performing image edge processing on the equalized image to obtain the edge image corresponding to the candidate image includes: performing wavelet decomposition on the equalized image to obtain sub-bands of different frequencies; estimating the coefficient variance of the sub-bands according to a wavelet threshold denoising strategy combined with a sliding window, and denoising the sub-bands according to the coefficient variance, and obtaining a denoised target processed image after wavelet reconstruction; calculating the gray-level gradient magnitude value corresponding to each pixel in the target processed image; constructing a membership function curve based on an S-function, the shape of which is jointly determined by the lower limit of the fuzzy interval, the midpoint of the fuzzy interval, and the upper limit of the fuzzy interval; using a genetic algorithm to iteratively optimize the lower limit of the fuzzy interval, the midpoint of the fuzzy interval, and the upper limit of the fuzzy interval to obtain optimal parameters; using the optimal parameters to determine the specific morphological function of the S-function, and calculating the corresponding edge membership degree in combination with the gray-level gradient magnitude value of each pixel; and performing edge annotation on the candidate image according to a preset membership degree threshold and the edge membership degree to obtain the corresponding edge image.

[0035] For example, a wavelet basis function such as the Daubechies wavelet is selected to perform multi-level wavelet decomposition on the input equalized image. After wavelet transform, it is decomposed into four sub-bands: one low-frequency sub-band and three high-frequency sub-bands. The low-frequency sub-band contains the main structural information and overall contour of the image, while the high-frequency sub-bands contain the detailed texture and noise information of the image.

[0036] For example, the median of the coefficients in the first high-frequency sub-band is used as the basis. All coefficients in the first high-frequency sub-band are taken as absolute values, sorted, and their medians are divided by a constant 0.6745 to obtain an estimate of the noise standard deviation. This estimate is then multiplied by a coefficient related to the number of decomposition layers and the window width to obtain the threshold for that sub-band at that layer. Then, the local variance, or coefficient variance, of each wavelet coefficient in each sub-band is estimated using a sliding window. For example, a fixed-size window is set on the current sub-band, and the window slides row by row and column by column on the sub-band. For each window position, all wavelet coefficients within the window are extracted, and their variances are calculated. This local variance reflects the signal strength in the region surrounding the coefficient. If the window contains real edge information, the coefficient variation will be larger, and the variance will be correspondingly larger; if the window mainly contains noise, the coefficient variation will be relatively smooth, and the variance will be smaller.

[0037] For example, coefficients at different locations are differentiated based on the aforementioned threshold and local variance. For a given wavelet coefficient, its absolute value is first determined to be greater than the threshold. If it is less than the threshold, it is directly set to zero and considered pure noise. If it is greater than the threshold, the local variance is further used to determine the signal probability of the coefficient. The larger the local variance, the more drastic the changes in the area surrounding the coefficient, and the higher the probability that it belongs to the real signal, thus the smaller the reduction in its amplitude. The smaller the local variance, the flatter the area surrounding the coefficient, and the higher the probability that it belongs to noise, thus the greater the reduction in its amplitude. After processing all sub-band coefficients, the wavelet coefficients are reconstructed, and the processed sub-band coefficients are merged layer by layer according to the inverse process of wavelet decomposition, finally obtaining the denoised target image.

[0038] For example, for each pixel in the target image, its gray-level change rate in the horizontal and vertical directions is calculated. Then, the square root of the square of the partial derivatives in these two directions is taken to obtain the gray-level gradient magnitude of that pixel. The gradient magnitude reflects the degree of gray-level change at that pixel location: in flat areas of the image, the gray-level values ​​of surrounding pixels do not change much, and the gradient magnitude is small; in edge areas of the image, the gray-level values ​​change drastically, and the gradient magnitude is large.

[0039] For example, the gradient magnitude of each pixel is converted into the probability or degree of belonging to an edge. This conversion is achieved using the S-function, also known as a sigmoid function. The S-function has an "S"-shaped curve, the specific shape of which is determined by three parameters: the lower limit of the fuzzy interval, the midpoint of the fuzzy interval, and the upper limit of the fuzzy interval. These three parameters together define a transition interval from "definitely not an edge" to "definitely an edge"—pixels with gradient magnitudes below the lower limit have a membership degree of 0 and are definitely not edges; pixels with gradient magnitudes above the upper limit have a membership degree of 1 and are definitely edges; pixels with gradient magnitudes between the lower and upper limits have a membership degree that smoothly transitions between 0 and 1, reflecting the gradual change from "non-edge" to "edge". By adjusting these three parameters, the shape of the S-function can be controlled: the distance between the lower and upper limits determines the width of the transition interval; the greater the distance, the smoother the transition; the smaller the distance, the steeper the transition; the position of the midpoint determines the bias of the transition interval; if the midpoint is to the left, more pixels are judged as edges, and if the midpoint is to the right, fewer pixels are judged as edges.

[0040] For example, the three parameters of the S-function—the lower bound of the fuzzy interval, the midpoint of the fuzzy interval, and the upper bound of the fuzzy interval—directly determine the accuracy of edge detection. Thus, a genetic algorithm is used to automatically find the optimal combination of parameters, thereby obtaining the optimal parameters.

[0041] For example, after obtaining the optimal parameters, they are substituted into the S-function to determine the specific form of the S-function. Then, the gray-level gradient magnitude of each pixel in the target image is input into the S-function one by one to calculate the edge membership degree corresponding to each pixel.

[0042] For example, a membership threshold is set, typically around 0.5. The edge membership of each pixel is compared with this threshold: if the edge membership of a pixel is greater than or equal to the threshold, the pixel is marked as an edge point; if the edge membership of a pixel is less than the threshold, the pixel is marked as a non-edge point. This yields the corresponding edge image.

[0043] Step S103: Use the front and back sides to try to filter the board image to obtain the corresponding valid drawings, and determine the corresponding tolerance parameters according to the board type corresponding to the target board.

[0044] For example, the purpose of drawing screening is to find the design drawing corresponding to the current board material to be tested from the existing drawing library. Since in actual production, there may be two drawings, one for the front and one for the back, it is necessary to determine the valid drawing by trying both sides.

[0045] For example, the current orientation can be determined by identifying the location of marks, labels, or specific process features on the surface of the board. Based on this determined orientation information, drawings matching that orientation are preferentially selected from the drawing library as candidate drawings. For instance, if the current detection is of the front of the board, front drawings are preferred; if the detection is of the back, back drawings are preferred. This prioritization strategy can significantly reduce the number of attempts and improve matching efficiency.

[0046] For example, tolerance parameters determine the leniency in the subsequent hole and slot matching process. Different types of sheet metal differ in processing accuracy, physical deformation, and inspection requirements; therefore, tolerance parameters need to be set according to the specific type of the sheet metal. Sheet metal types include normal sheets, short sheets, and beveled sheets. Based on the sheet metal type, the corresponding tolerance parameters are read from the preset parameter configuration. Tolerance parameters typically include: center deviation threshold (the maximum allowable distance deviation between the measured hole center and the hole center on the drawing); hole diameter tolerance (the allowable range of difference between the measured inner and outer diameters of the hole and the nominal hole diameter on the drawing); slot position deviation threshold (the maximum allowable coordinate deviation between the measured slot end point and the slot end point on the drawing), etc.

[0047] Step S104: Determine the drawing coordinate system corresponding to the valid drawing and the image coordinate system corresponding to the candidate image, and establish an affine matrix from the candidate image to the valid drawing based on the drawing coordinate system and the image coordinate system.

[0048] For example, corner points or reference marks of the board material are identified in the image, and their pixel coordinates in the image coordinate system are obtained. At the same time, based on the drawing size information, the theoretical physical coordinates of these points in the drawing coordinate system are determined, thus forming multiple sets of corresponding point pairs. Based on these point pairs, the six parameters of the affine transformation, including rotation, scaling, translation, and shear components, are solved using mathematical methods such as the least squares method, and finally, an affine matrix from the image coordinate system to the drawing coordinate system is constructed.

[0049] Step S105: Perform geometric analysis on the measured hole area and the slot candidate area to obtain initial geometric features, and perform information transformation on the initial geometric features according to the affine matrix to obtain the target geometric features corresponding to the drawing coordinate system.

[0050] For example, the edge information of the measured hole area and the slot candidate area are fitted respectively to obtain the geometric features corresponding to the circle in the measured hole area and the geometric features corresponding to the L-shaped in the slot candidate area, which together form the initial geometric features.

[0051] For example, edge points of the measured hole area are fitted to obtain its center position and the range of inner and outer diameters reflecting its dimensions; for candidate slot areas, their type is determined by analyzing the combination relationship between edges in their contours. For example, an L-shaped structure formed by two edges intersecting at right angles is identified, and parameters such as the turning point position, the length of the two arms, and the included angle are extracted. The feature parameters of the holes and slots of different shapes are summarized to form the initial geometric features.

[0052] For example, using the established affine matrix, all key parameters of the initial geometric features are mapped from the image coordinate system to the drawing coordinate system, thereby obtaining the target geometric features corresponding to the drawing coordinate system.

[0053] In some embodiments, the step of performing geometric analysis on the measured hole region and the slot candidate region to obtain initial geometric features includes: simultaneously calculating the maximum inscribed circle and the minimum circumscribed circle for the measured hole region, and determining the center position and diameter information of the target hole in the measured hole region based on the maximum inscribed circle and the minimum circumscribed circle; determining the length and width information of the target slot in the slot candidate region based on the minimum rotated circumscribed rectangle of the slot candidate region, and determining the direction information of the target slot based on the length and width information; extracting two diagonal endpoints from the slot candidate region to describe the position of the slot corresponding to the target slot based on the length and width information combined with the direction information; and determining the corresponding initial geometric features based on the center position and diameter information of the target hole and the direction information and diagonal endpoints of the target slot.

[0054] For example, the maximum inscribed circle and minimum circumscribed circle are fitted and calculated simultaneously for the measured hole area. The maximum inscribed circle is the largest complete circle that can be accommodated within the hole area; its diameter reflects the effective bore diameter, and its center represents the center of the hole. The minimum circumscribed circle is the smallest circle that completely encloses the entire hole area; its diameter reflects the maximum contour range of the target hole, and its center also points to the center of the target hole. By combining the center positions of the maximum inscribed circle and the minimum circumscribed circle, and after weighted or averaged processing, the center coordinates of the target hole can be determined more stably, ensuring the accuracy of center positioning even if the target hole has slight deformation or edge defects. The inner and outer diameter ranges of the target hole are formed by combining the diameter information of the two circles.

[0055] For example, for a candidate slot region, the minimum rotational bounding rectangle is calculated, which is the smallest area rectangle that can enclose the entire slot region at any angle. The direction of the long side of this rectangle is the extension direction of the target slot, the length of the long side corresponds to the length of the target slot, and the length of the short side corresponds to the width of the target slot. This determines the length and width information of the target slot, and simultaneously determines the orientation information of the target slot based on the orientation of the long side. Based on this, along the extension direction of the target slot, i.e., the direction of the long side of the rectangle, the two diagonal endpoints at both ends of the target slot are extracted. These two endpoints are located at the beginning and end of the target slot, respectively, and are used to describe the spatial position of the target slot. Since matching the target slot requires not only determining its approximate position but also ensuring correct orientation, using two endpoints instead of a single center point to describe the target slot position can simultaneously carry both position and orientation information.

[0056] For example, the center position, inner and outer diameter range of the target hole, the direction information, two diagonal endpoints, length and width of the target groove are summarized to form the initial geometric features.

[0057] For example, using the established affine matrix, all key parameters of the initial geometric features are mapped from the image coordinate system to the drawing coordinate system, thereby obtaining the target geometric features corresponding to the drawing coordinate system. The process for obtaining the target geometric features is as follows: Figure 2 As shown, Step S106: Perform attitude traversal based on the valid drawing and the target geometric features, and combine the tolerance parameters to match the measured hole and slot with the designed hole and slot in the valid drawing to obtain the target matching result.

[0058] For example, by traversing the candidate drawings and their rotational attitudes, the measured holes and slots are matched one by one with the theoretical holes and slots in the design drawings, and multiple evaluation indicators are calculated based on the matching results. Finally, the drawing and attitude with the highest matching degree with the current board material are determined to obtain the target matching result.

[0059] For example, the first candidate drawing in the valid drawings is first tested with four rotational orientations: 0°, 90°, 180°, and 270°. Hole matching and slot matching are performed in each orientation. If the first drawing fails to meet the set matching requirements in any orientation and a second candidate drawing exists, the process switches to the second drawing and repeats the search process for the four orientations. The entire matching loop is executed a maximum of eight times.

[0060] For example, for each measured hole, a nearest neighbor search with hole diameter constraints is performed within the range of design holes that have not yet been matched in the current drawing. After meeting the hole diameter tolerance requirements in the tolerance parameters, the hole with the closest center distance is selected as the matching result. If a match is successful, the hole position matching count is updated and the corresponding drawing hole is marked as matched. Otherwise, if no matching drawing hole is found for the measured hole, the search is unsuccessful.

[0061] For example, the orientation of the measured groove is first determined based on the difference in side lengths in the X and Y directions, classifying it into a transverse groove or a longitudinal groove. For a transverse groove, the Y coordinates of the two endpoints of the measured groove are compared with the Y coordinates of the corresponding endpoints of the groove on the drawing; the deviations of both must meet the set groove position deviation threshold. For a longitudinal groove, the X coordinates of the two endpoints are compared. A drawing groove whose coordinate deviations are all within the threshold range is considered a successful match, marked as matched, and the groove matching count is updated. If no matching drawing groove is found, the measured groove is classified into the groove defect area and subsequently determined as multiple grooves or groove position errors.

[0062] For example, after matching holes and slots in each drawing pose is completed, three evaluation metrics are calculated: strict coverage rate, reflecting the proportion of ordinary holes and slots in the drawing that are all found; full coverage rate, reflecting the proportion of all holes and slots in the drawing that are all found; and measured matching rate, reflecting the proportion of holes and slots in the actual drawing that are successfully parsed by the drawing, used to assist in identifying multiple holes, multiple slots, and positional offset issues. When selecting the best pose, the pose with the highest strict coverage rate is prioritized; if multiple poses have the same strict coverage rate, the pose with the highest measured matching rate is selected. Finally, the best matching drawing, rotation angle, values ​​of each evaluation metric, and corresponding hole and slot defect area information are output, forming a complete target matching result.

[0063] In some implementations, the step of performing attitude traversal based on the valid drawings and the target geometric features, and combining the tolerance parameters to match the measured hole slots with the designed hole slots in the valid drawings to obtain the target matching result, includes: obtaining the design hole information of the first candidate drawing in the valid drawings in the current attitude; filtering out the design hole information of the first candidate drawing whose hole marking information is in the current attitude as hole mismatch state, as the hole information to be matched; traversing the hole information to be matched, calculating the center distance between the center coordinates of the hole information to be matched and the center position of the measured target hole, and calculating the absolute value of the difference between the diameter information of the target hole and the nominal diameter of the hole information to be matched; When at least one of the holes to be matched simultaneously satisfies the condition that the center distance is less than or equal to the center deviation threshold and the absolute value of the difference is less than or equal to the diameter deviation threshold, the hole with the smallest center distance is selected from all the holes to be matched that meet the conditions as the matching hole corresponding to the target hole. The first target state of the target hole is marked as the hole normal state, and the hole marking information corresponding to the hole to be matched is modified from the hole unmatched state to the hole matched state. When no hole to be matched simultaneously satisfies the condition that the center distance is less than or equal to the center deviation threshold and the absolute value of the difference is less than or equal to the diameter deviation threshold, it is checked whether the center position of the target hole falls within the pre-marked label occlusion area. Within the polygon, if yes, the first target state corresponding to the target hole is marked as the normal state of the hole; if no, check whether the diameter information of the target hole reaches a preset effective size threshold. If yes, the first target state corresponding to the target hole is marked as the normal state of the hole; if no, calculate the actual roundness of the target hole, and if the actual roundness is greater than or equal to a preset roundness threshold, mark the first target state corresponding to the target hole as the normal state of the hole; otherwise, mark the first target state corresponding to the target hole as the abnormal state of the hole. Repeat the above steps until all the target holes under the current posture have been traversed to obtain the first target state corresponding to all the target holes; the target groove Geometric matching is performed on the first candidate drawing in its current pose to obtain the information matching result corresponding to the target slot; the matching index corresponding to the target hole in the first candidate drawing in its current pose is calculated based on the first target state and the information matching result, the matching index including strict coverage and measured matching rate; the above steps are repeated, the pose of the first candidate drawing is adjusted and matching and index calculation are performed again; if the first candidate drawing does not meet the preset matching requirements in all poses and there is a second candidate drawing, the second candidate drawing is switched and the above matching process is repeated; among all the poses of the drawings that have completed matching, the pose with the highest strict coverage is selected as the best matching result;If multiple drawing poses have the same strict coverage rate, the pose with the highest measured matching rate is selected as the best matching result. Based on the best matching result, the target matching result between the measured hole / groove and the designed hole / groove in the valid drawing is determined. The target matching result includes the drawing number, the drawing pose, and the matching index.

[0064] For example, in order to achieve a precise match between the measured hole and slot and the design drawing, hole position matching, slot position matching, index calculation and attitude optimization are performed in sequence.

[0065] For example, in the hole matching stage, firstly, all design hole information of the current candidate drawing in the current posture is obtained, and the design holes marked as "hole not matched" are selected as the hole information to be matched. For each target hole, the above hole information to be matched is traversed, and the center distance between the center coordinates of the target hole and the center coordinates of each hole information to be matched is calculated. At the same time, the absolute value of the difference between the diameter information of the target hole and the nominal diameter of the hole information to be matched is calculated. If there is at least one hole information to be matched that simultaneously satisfies "center distance less than or equal to the center deviation threshold of the tolerance parameter" and "absolute value of diameter difference less than or equal to the diameter deviation threshold of the tolerance parameter", then the hole with the smallest center distance among all the hole information to be matched that meets the conditions is selected as the matching hole of the target hole, the first target state of the target hole is marked as "hole normal state", and the marking information of the corresponding hole information to be matched is updated from "hole not matched state" to "hole matched state". If no matching hole simultaneously meets both of the above constraints, the re-judgment process begins: First, check whether the center position of the target hole falls within the pre-marked label occlusion area polygon. If so, the first target state corresponding to the target hole is marked as the normal state. If not, further check whether the diameter information of the target hole reaches the preset effective size threshold. If so, the first target state corresponding to the target hole is marked as the normal state. If not, calculate the actual roundness of the target hole. If the actual roundness is greater than or equal to the preset roundness threshold, the first target state corresponding to the target hole is marked as the normal state. Otherwise, the first target state corresponding to the target hole is marked as the abnormal state.

[0066] For example, the above process is repeated until all the measured target holes under the current attitude are traversed to obtain the first target state of each target hole.

[0067] For example, in the slot matching stage, the target slot is geometrically matched to the current pose of the first candidate drawing. The matching is classified according to the direction of the slot: the Y coordinate deviation of the two ends of the transverse slot is matched, and the X coordinate deviation of the two ends of the longitudinal slot is matched. If the deviation is within a set threshold, the matching is successful; otherwise, the matching fails, thus obtaining the information matching result.

[0068] For example, after obtaining the first target state of all target holes and the information matching results of target slots, the matching index under the current attitude is calculated, which mainly includes: strict coverage and measured matching rate.

[0069] For example, after completing the matching and index calculation for the current pose, the first candidate drawing is rotated to the next angle, and the entire matching and calculation process is repeated until all poses such as 0°, 90°, 180°, and 270° are completed. If the first candidate drawing fails to meet the preset matching requirements in all poses and a second candidate drawing exists, the process is switched to the second candidate drawing and the entire matching process for all poses is repeated.

[0070] For example, after all drawing poses are matched, the pose with the highest strict coverage rate is selected as the best matching result. If multiple poses have the same strict coverage rate, the pose with the highest measured matching rate is selected as the best matching result. Finally, based on the selected best matching result, the target matching result between the measured hole / groove and the designed hole / groove in the valid drawing is determined. The output includes the drawing number used for matching, the best drawing pose, and the corresponding matching indicators.

[0071] In some implementations, the step of performing geometric matching on the target slot in the current pose of the first candidate drawing to obtain the information matching result corresponding to the target slot includes: obtaining the coordinate information and first slot direction corresponding to all design slots in the current pose of the first candidate drawing; classifying the design slots according to the first slot direction to obtain first slot information and second slot information, wherein the first slot information represents all design slots whose first slot direction is a transverse slot, and the second slot information represents all design slots whose first slot direction is a longitudinal slot; determining the direction according to the direction information corresponding to the target slot to determine the second slot direction corresponding to the target slot, wherein the second slot direction is the transverse slot. And any of the longitudinal slots; when the direction of the second slot is the transverse slot, traverse the first sub-slots in the first slot information, calculate the difference between the first longitudinal coordinates of the two diagonal endpoints of the target slot and the second longitudinal coordinates of the two endpoints of the first sub-slot, and obtain the first coordinate difference values ​​corresponding to the two endpoints respectively; when both first coordinate differences are less than or equal to the slot deviation threshold in the tolerance parameter, mark the second target state of the target slot as the slot normal state, mark the slot marking information corresponding to the first sub-slot as the slot matched state, and end the search for the current target slot, and continue to process the next target slot; after traversing all the first sub-slots in the first slot information, if no one If the difference between the two first coordinates between any first sub-slot and the target slot is less than or equal to the slot deviation threshold, then the second target state of the target slot is marked as a slot abnormal state, and the next target slot is processed. When the second slot direction is the longitudinal slot, the second sub-slots in the second slot information are traversed, and the difference between the first lateral coordinates of the two diagonal endpoints of the target slot and the second lateral coordinates of the two endpoints of the second sub-slot is calculated to obtain the second coordinate difference corresponding to the two endpoints. When the difference between the two second coordinates is less than or equal to the slot deviation threshold, the second target state of the target slot is marked as the slot normal state, and the second sub-slot is... The corresponding slot marking information is marked as the slot matched, and the search for the current target slot ends, and the next target slot is processed. After traversing all the second sub-slots in the second slot information, if there is no second sub-slot whose two second coordinate differences with the target slot are both less than or equal to the slot deviation threshold, the second target state of the target slot is marked as the slot abnormal state, and the next target slot is processed. The above steps are repeated until the second target state of all the target slots is obtained. When performing each target slot matching with the first sub-slot or the second sub-slot, the design slots whose slot marking information has been marked as the slot matched are skipped.Data processing is performed on all the target slots according to their second target states to obtain the information matching result corresponding to the target slot in the current pose of the first candidate drawing.

[0072] For example, when performing slot matching in the current orientation of the first candidate drawing, the coordinate information of all design slots on the first candidate drawing and their corresponding first slot directions are obtained. Based on the first slot direction, all design slots are divided into two categories: the first slot information includes all horizontal design slots, and the second slot information includes all vertical design slots.

[0073] For example, for each target slot to be matched, the direction is determined based on its own geometric orientation to determine whether the second slot direction is a transverse slot or a longitudinal slot. If the target slot is a transverse slot, each first sub-slot is traversed from the first slot information, and the difference between the first longitudinal coordinates of the two diagonal endpoints of the target slot and the second longitudinal coordinates of the two endpoints of the current first sub-slot is calculated to obtain the first coordinate difference corresponding to the two endpoints. When the first coordinate difference of the two endpoints is less than or equal to a preset slot deviation threshold, it is determined that the target slot is successfully matched with the current design slot, the second target state of the target slot is marked as the slot normal state, and the slot marking information corresponding to the first sub-slot is updated to the slot matched state. Then, the search for the current target slot ends and the next target slot is processed. If no matching object is found after traversing all transverse design slots and the coordinate deviation of the two endpoints does not exceed the threshold, the second target state of the target slot is marked as the slot abnormal state, and the next target slot is processed.

[0074] For example, if the target slot is a longitudinal slot, the matching logic is symmetrical with that of the transverse slot: traverse each second sub-slot from the second slot information, calculate the difference between the first transverse coordinates of the two diagonal endpoints of the target slot and the second transverse coordinates of the two endpoints of the current second sub-slot, and obtain the difference in the second coordinates corresponding to the two endpoints. When the difference in the second coordinates of the two endpoints is less than or equal to the slot deviation threshold, the match is marked as successful, the target slot status is set to the normal slot status, and the corresponding design slot is marked as matched. The current search ends, and the next target slot is processed. If no matching condition is found after traversing all longitudinal design slots, the target slot is marked as the abnormal slot status, and the next target slot is processed.

[0075] For example, during the entire matching process, before performing the matching of each target slot with the design slot, design slots whose slot marking information has been marked as matched are skipped to ensure that each design slot matches only one actual slot and avoid duplicate matching.

[0076] For example, the above process is repeated until all target slots obtain a second target state (slot normal state or slot abnormal state). Finally, statistical processing is performed based on the second target states of all target slots to obtain the information matching result under the current posture. The information matching result includes the number of successfully matched slots (target slots in the normal state) and the number of unmatched slot defects (target slots in the abnormal state), as well as the second target state corresponding to each target slot, providing a data basis for subsequent matching index calculations.

[0077] For example, such as Figure 3 As shown, this scheme searches and evaluates candidate drawings and their rotational attitudes through four steps: attitude traversal, hole and slot matching, matching rate calculation, and optimal attitude selection. Starting with the first drawing, four rotational attitudes—0°, 90°, 180°, and 270°—are tried sequentially. Hole and slot matching are performed for each attitude. If all attitudes on the first drawing fail and a second drawing exists, the same search is performed on the second drawing. The entire loop is repeated a maximum of 8 times (2 drawings × 4 attitudes). During hole matching, the center distance, center deviation threshold, and inner and outer diameter tolerances of each measured hole are considered when no matching is found on the drawing. In hole matching, a nearest neighbor search with hole diameter constraints is performed. If a match is successful, the hole in the drawing is marked and the number is accumulated. If a match fails, it is classified as multiple holes or incorrect hole positions after verification under specific conditions. For slot matching, the transverse or longitudinal slot is determined based on the side length in the X and Y directions, and the deviation of the endpoint Y coordinate or X coordinate is constrained respectively. If a match is successful, the slot in the drawing is marked. If a match fails, it is classified as multiple slots or incorrect slot positions. After each posture matching is completed, the strict coverage rate, the full coverage rate, and the measured matching rate are calculated. Finally, the drawing posture with the highest strict coverage rate is selected first. If they are the same, the posture with the higher measured matching rate is selected, and the best drawing, rotation angle, various indicators, and corresponding defect areas are output.

[0078] Step S107: Based on the target matching result and the defect candidate area, perform result analysis and defect determination to obtain the target defect information corresponding to the target board.

[0079] For example, based on the matching indicators and hole / groove matching status in the target matching results, the number of valid measured holes / grooves, the number of holes / grooves that can be matched on the drawings, and the number of unmatched holes / grooves are statistically analyzed. By comparing these quantitative relationships, a preliminary judgment is made as to whether there are defect trends such as multiple holes, leaking holes, multiple grooves, or leaking grooves. The judgment results of hole-type defects and groove-type defects are summarized to form a complete defect information list, which includes the type and location information of each defect, serving as the final output target defect information. The chipping candidate area serves as auxiliary evidence in the entire judgment process to participate in the re-judgment screening of leaking holes, helping to eliminate false detections caused by chipping at the hole edge and improving the accuracy of defect judgment.

[0080] In some implementations, obtaining target defect information corresponding to the target board material by analyzing the results and determining defects based on the target matching results and the candidate chipping areas includes: determining the target drawing based on the drawing number and the drawing orientation, and determining the number of valid measured holes, the number of holes that can be matched on the drawing, and the number of unmatched holes based on the matching index and the target drawing; if the number of valid measured holes is greater than the number of holes that can be matched on the drawing, then the hole defect type corresponding to the target board material is determined to be a porous defect; if the number of valid measured holes is less than the number of holes that can be matched on the drawing, then the hole defect type is determined to be a leaky hole defect; if If the number of effective measured holes equals the number of holes that can be matched on the drawing, but the number of unmatched holes is greater than a preset number, then the hole defect type is determined to be hole position deviation. The relevant geometric features corresponding to the leak information under the number of unmatched holes are compared with the candidate area for breakage. If the relevant distance between the relevant geometric features and the candidate area for breakage is less than or equal to a preset distance, then the first target state corresponding to the leak information is determined as the normal state of the hole. If the relevant distance is greater than the preset distance, then first local grayscale information is obtained from the candidate image based on the relevant geometric features, and the first local grayscale information is obtained from the candidate image. The second local grayscale information corresponding to the background surrounding the relevant geometric features is used to compare the brightness difference between the first local grayscale information and the second local grayscale information. When the brightness difference is greater than a preset brightness, the first target state corresponding to the hole information is determined as the normal state of the hole; otherwise, the first target state corresponding to the hole information is determined as the abnormal state of the hole. Based on the matching index and the target drawing, the number of valid measured grooves, the number of grooves that can be matched in the drawing, and the number of unmatched grooves are determined. When the number of valid measured grooves is greater than the number of grooves that can be matched in the drawing, the groove defect type corresponding to the target board is determined to be... Multiple slot defects; when the number of effective measured slots is less than the number of slots that can be matched on the drawing, and there are design slots whose slot marking information is not marked as matching, the slot defect type is determined to be a missing slot defect; when the number of effective measured slots is equal to the number of slots that can be matched on the drawing, but the endpoint coordinate deviation between the target slot and the design slot exceeds a set threshold, the slot defect type is determined to be a slot misalignment defect; the target defect information corresponding to the target plate is determined according to the slot defect type, the hole defect type, and the first target state corresponding to the target hole and the second target state corresponding to the target slot.

[0081] For example, the target drawing is determined based on the selected drawing number and drawing orientation. Then, three key data points are statistically analyzed using matching indicators: the number of valid measured holes (the total number of reliable measured holes retained after preprocessing), the number of holes that can be matched with the drawing (the total number of holes included in the assessment in the target drawing), and the number of unmatched holes (the number of measured holes that could not be matched with the holes in the drawing). The type of hole defect is determined by comparing the relationship between the above three data points: if the number of valid measured holes is greater than the number of holes that can be matched with the drawing, it is determined to be a multi-hole defect; if the number of valid measured holes is less than the number of holes that can be matched with the drawing, it is determined to be a leaky hole defect; if the number of valid measured holes is equal to the number of holes that can be matched with the drawing but the number of unmatched holes is greater than a preset number, it is determined to be a hole position deviation.

[0082] For example, for each candidate leak in the leak defect, a second judgment is made in conjunction with the candidate area for broken edges to eliminate false detections caused by broken edges or image noise. The relevant geometric features of the candidate leak are compared with the position of the candidate area for broken edges. If the spatial distance between the two is less than or equal to a preset distance, it indicates that there is a broken edge near the leak. The failure to identify the leak may be due to incomplete contours caused by the broken edge. Therefore, the first target state of the leak is corrected to the normal state. If the relevant distance is greater than the preset distance, the first local grayscale information of the leak location is further extracted from the candidate image, and the brightness difference is compared with the second local grayscale information of the surrounding background area. If the brightness difference is greater than a preset brightness threshold, it indicates that the model may have missed the detection of the location due to local darkness or brightness. The leak is also corrected to the normal state. If the brightness difference does not exceed the threshold, the leak is confirmed to be a real abnormal state.

[0083] For example, the number of valid measured slots, the number of slots that can be matched on the drawing, and the number of unmatched slots are determined based on the matching index and the target drawing. When the number of valid measured slots is greater than the number of slots that can be matched on the drawing, it is judged as a multi-slot defect; when the number of valid measured slots is less than the number of slots that can be matched on the drawing, and there are design slots whose slot marking information is not marked as slot matched, it is judged as a missing slot defect; when the number of valid measured slots is equal to the number of slots that can be matched on the drawing, but the endpoint coordinate deviation between the target slot and the design slot exceeds a set threshold, it is judged as a slot misalignment defect.

[0084] For example, the above-mentioned hole defect types and their corresponding specific geometric information, groove defect types and their corresponding specific geometric information, as well as the first target state of each hole and the second target state of each groove after re-judgment and correction, are summarized and integrated to generate complete target defect information of the target plate, which is the final output result.

[0085] Please see Figure 4 , Figure 4This application provides a flexible detection system 200 for hole and groove defects in finished sheet metal. The system includes a region cropping module 201, an image processing module 202, a drawing selection module 203, a coordinate transformation module 204, a feature processing module 205, an information matching module 206, and a defect determination module 207. The region cropping module 201 is used to crop candidate regions based on the effective range of the target sheet metal in the sheet metal image to obtain candidate images. The image processing module 202 is used to preprocess and perform edge contour processing on the candidate images to obtain corresponding edge images, and to filter and analyze the candidate images based on the edge images to obtain corresponding target candidate regions in the candidate images. The target candidate regions include measured hole regions, chipping candidate regions, and groove candidate regions. The drawing selection module 203 is used to attempt to filter the sheet metal image using both front and back views to obtain corresponding valid drawings, and to determine the target... The tolerance parameters are determined based on the type of board material corresponding to the board material. A coordinate transformation module 204 is used to determine the drawing coordinate system corresponding to the valid drawing and the image coordinate system corresponding to the candidate image, and to establish an affine matrix from the candidate image to the valid drawing based on the drawing coordinate system and the image coordinate system. A feature processing module 205 is used to perform geometric analysis on the measured hole area and the slot candidate area to obtain initial geometric features, and to perform information transformation on the initial geometric features based on the affine matrix to obtain the target geometric features corresponding to the drawing coordinate system. An information matching module 206 is used to perform attitude traversal based on the valid drawing and the target geometric features, and to combine the tolerance parameters to match the measured holes and slots with the designed holes and slots in the valid drawing to obtain the target matching result. A defect determination module 207 is used to perform result analysis and defect judgment based on the target matching result and the chipping candidate area to obtain the target defect information corresponding to the target board material.

[0086] In some implementations, the flexible detection system 200 for hole and groove defects in finished sheet metal can be applied to terminal equipment.

[0087] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the flexible detection system 200 for hole and groove defects in finished sheet metal described above can be referred to the corresponding process in the aforementioned embodiments of the flexible detection method for hole and groove defects in finished sheet metal, and will not be repeated here.

[0088] Please see Figure 5 , Figure 5 This is a schematic block diagram of the structure of a terminal device provided in an embodiment of the present invention.

[0089] like Figure 5As shown, the terminal device 300 includes a processor 301 and a memory 302, which are connected via a bus 303, such as an I2C (Inter-integrated Circuit) bus.

[0090] Specifically, processor 301 provides computing and control capabilities to support the operation of the entire terminal device. Processor 301 can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0091] Specifically, the memory 302 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a portable hard drive, etc.

[0092] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the embodiments of the present invention, and does not constitute a limitation on the terminal device to which the embodiments of the present invention are applied. A specific server may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0093] The processor is used to run a computer program stored in a memory, and when executing the computer program, implements any of the flexible detection methods for hole and groove defects in finished sheet metal provided in the embodiments of the present invention.

[0094] In one embodiment, the processor is configured to run a computer program stored in memory, and when executing the computer program, perform the following steps: Candidate images are obtained by cropping the candidate region based on the effective range of the target board in the board image; The candidate image is preprocessed and edge contour processing is performed to obtain the corresponding edge image. The candidate image is then filtered and analyzed based on the edge image to obtain the corresponding target candidate region in the candidate image. The target candidate region includes the measured hole region, the chipping candidate region, and the slot candidate region. By using both the front and back sides to attempt to filter the images of the board material to obtain the corresponding valid drawings, and by determining the corresponding tolerance parameters according to the board type corresponding to the target board material; Determine the drawing coordinate system corresponding to the valid drawing and the image coordinate system corresponding to the candidate image, and establish an affine matrix from the candidate image to the valid drawing based on the drawing coordinate system and the image coordinate system; Geometric analysis is performed on the measured hole area and the slot candidate area to obtain initial geometric features, and the initial geometric features are transformed according to the affine matrix to obtain the target geometric features corresponding to the drawing coordinate system. The attitude traversal is performed based on the valid drawings and the target geometric features, and the measured holes and slots are matched with the designed holes and slots in the valid drawings to obtain the target matching result by combining the tolerance parameters. Based on the target matching results and the defect candidate regions, result analysis and defect determination are performed to obtain the target defect information corresponding to the target board.

[0095] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the terminal equipment described above can be referred to the corresponding process in the aforementioned embodiment of the flexible detection method for hole and groove defects in finished sheet metal, and will not be repeated here.

[0096] This invention also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of any of the flexible detection methods for hole and groove defects in finished sheet metal as provided in the specification of this invention.

[0097] The storage medium can be an internal storage unit of the terminal device described in the foregoing embodiments, such as the hard drive or memory of the terminal device. Alternatively, the storage medium can be an external storage device of the terminal device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the terminal device.

[0098] Those skilled in the art will understand that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware embodiments, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0099] It should be understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. It should be noted that, herein, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0100] The sequence numbers of the above embodiments of the present invention are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The above descriptions are only specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for flexible detection of hole and groove defects in finished sheet metal, characterized in that, include: Candidate images are obtained by cropping the candidate region based on the effective range of the target board in the board image; The candidate image is preprocessed and edge contour processing is performed to obtain the corresponding edge image. The candidate image is then filtered and analyzed based on the edge image to obtain the corresponding target candidate region in the candidate image. The target candidate region includes the measured hole region, the chipping candidate region, and the slot candidate region. By using both the front and back sides to attempt to filter the images of the board material to obtain the corresponding valid drawings, and by determining the corresponding tolerance parameters according to the board type corresponding to the target board material; Determine the drawing coordinate system corresponding to the valid drawing and the image coordinate system corresponding to the candidate image, and establish an affine matrix from the candidate image to the valid drawing based on the drawing coordinate system and the image coordinate system; Geometric analysis is performed on the measured hole area and the slot candidate area to obtain initial geometric features, and the initial geometric features are transformed according to the affine matrix to obtain the target geometric features corresponding to the drawing coordinate system. The attitude traversal is performed based on the valid drawings and the target geometric features, and the measured holes and slots are matched with the designed holes and slots in the valid drawings to obtain the target matching result by combining the tolerance parameters. Based on the target matching results and the defect candidate regions, result analysis and defect determination are performed to obtain the target defect information corresponding to the target board.

2. The method according to claim 1, characterized in that, The step of preprocessing and edge contour processing the candidate image to obtain the corresponding edge image includes: The candidate image is denoised using a bilateral filter that combines spatial proximity weights and color proximity weights to obtain a denoised image. Set a segmentation threshold, and determine the effective brightness range corresponding to the denoised image based on the segmentation threshold; The target segmentation point is determined based on the effective brightness range, and the histogram corresponding to the denoised image is divided into a left sub-histogram and a right sub-histogram based on the target segmentation point. The first brightness mapping range corresponding to the first sub-region of the left sub-histogram and the second brightness mapping range corresponding to the second sub-region of the right sub-histogram are dynamically adjusted according to the distribution weight of the left sub-histogram and the right sub-histogram. The brightness of the denoised image is adjusted according to the first brightness mapping range and the second brightness mapping range to obtain a brightness-adjusted brightness image. The exponential factor of the gamma correction algorithm is dynamically adjusted based on the average pixel value of the brightness image as a control benchmark, and the brightness image is subjected to contrast equalization processing based on the exponential factor and the gamma correction algorithm to obtain an equalized image. The edge image corresponding to the candidate image is obtained by performing image edge processing on the equalized image.

3. The method according to claim 2, characterized in that, The step of performing image edge processing on the equalized image to obtain the edge image corresponding to the candidate image includes: Wavelet decomposition is performed on the equalized image to obtain sub-bands of different frequencies; The coefficient variance of the sub-band is estimated by combining the wavelet threshold denoising strategy with the sliding window, and the sub-band is denoised according to the coefficient variance. After wavelet reconstruction, the denoised target image is obtained. Calculate the grayscale gradient magnitude of each pixel in the target image; construct a membership function curve based on the S-function, the shape of which is determined by the lower limit of the fuzzy interval, the midpoint of the fuzzy interval, and the upper limit of the fuzzy interval. The optimal parameters are obtained by iteratively optimizing the lower limit of the fuzzy interval, the midpoint of the fuzzy interval, and the upper limit of the fuzzy interval using a genetic algorithm. The specific shape function of the S-function is determined using the optimal parameters, and the corresponding edge membership degree is calculated by combining the gray-level gradient magnitude of each pixel. The candidate image is edge-labeled based on a preset membership threshold and the edge membership, and the corresponding edge image is obtained.

4. The method according to claim 1, characterized in that, The step of performing geometric analysis on the measured hole region and the candidate slot region to obtain initial geometric features includes: The maximum inscribed circle and the minimum circumscribed circle are calculated simultaneously for the measured hole area, and the center position and diameter information of the target hole in the measured hole area are determined based on the maximum inscribed circle and the minimum circumscribed circle. The length and width information of the target slot in the slot candidate region are determined based on the minimum rotational bounding rectangle of the slot candidate region, and the orientation information of the target slot is determined based on the length and width information. Based on the length information, the width information, and the direction information, two diagonal endpoints used to describe the position of the corresponding slot of the target slot are extracted from the slot candidate region; The initial geometric features are determined based on the center position and diameter information of the target hole, the direction information and diagonal endpoints of the target groove.

5. The method according to claim 4, characterized in that, The step of performing attitude traversal based on the valid drawings and the target geometric features, and combining the tolerance parameters to match the measured hole slots with the designed hole slots in the valid drawings to obtain the target matching result includes: Obtain the design hole information of the first candidate drawing in the valid drawings under the current posture; The design hole information in the first candidate drawing where the hole marking information is in a hole mismatched state under the current posture is selected as the hole information to be matched. Traverse the hole information to be matched, calculate the center distance between the center coordinates of the hole information to be matched and the center position of the measured target hole, and calculate the absolute value of the difference between the diameter information of the target hole and the nominal diameter of the hole information to be matched. When at least one of the holes to be matched simultaneously satisfies that the center distance is less than or equal to the center deviation threshold and the absolute value of the difference is less than or equal to the diameter deviation threshold, the hole with the smallest center distance is selected from all the holes to be matched that meet the conditions as the matching hole corresponding to the target hole, the first target state of the target hole is marked as the normal state of the hole, and the hole marking information corresponding to the hole to be matched is modified from the unmatched state to the matched state. When no matching hole information exists that simultaneously satisfies the condition that the center distance is less than or equal to the center deviation threshold and the absolute value of the difference is less than or equal to the diameter deviation threshold, check whether the center position of the target hole falls within a pre-defined label occlusion area polygon. If yes, the first target state corresponding to the target hole is marked as the hole normal state. If no, check whether the diameter information of the target hole reaches a preset effective size threshold. If yes, the first target state corresponding to the target hole is marked as the hole normal state. If no, calculate the actual roundness of the target hole, and if the actual roundness is greater than or equal to a preset roundness threshold, mark the first target state corresponding to the target hole as the hole normal state; otherwise, mark the first target state corresponding to the target hole as the hole abnormal state. Repeat the above steps until all the target holes in the current posture have been traversed, and obtain the first target state corresponding to all the target holes; The target slot is geometrically matched in the current pose of the first candidate drawing to obtain the information matching result corresponding to the target slot; The matching index of the target hole in the current posture of the first candidate drawing is calculated based on the first target state and the information matching result. The matching index includes strict coverage and measured matching rate. Repeat the above steps, adjust the pose of the first candidate drawing and perform matching and index calculation again; if the first candidate drawing fails to meet the preset matching requirements under all poses and there is a second candidate drawing, switch to the second candidate drawing and repeat all the above matching processes. Among all the matching drawing poses, the pose with the highest strict coverage rate is selected as the best matching result; if multiple drawing poses have the same strict coverage rate, the pose with the highest measured matching rate is selected as the best matching result. The target matching result between the measured hole and slot and the designed hole and slot in the valid drawing is determined based on the best matching result. The target matching result includes the drawing number, the drawing orientation, and the matching index.

6. The method according to claim 5, characterized in that, The step of performing geometric matching of the target slot in the current pose of the first candidate drawing to obtain the information matching result corresponding to the target slot includes: Under the current posture of the first candidate drawing, obtain the coordinate information of all design slots and the direction of the first slot; The design slots are classified according to the first slot direction to obtain first slot information and second slot information, wherein the first slot information represents all design slots whose first slot direction is a transverse slot, and the second slot information represents all design slots whose first slot direction is a longitudinal slot. Based on the direction information corresponding to the target slot, the direction is determined to identify the second slot direction corresponding to the target slot, where the second slot direction is either the transverse slot or the longitudinal slot. When the second slot direction is the transverse slot, the first sub-slot in the first slot information is traversed, and the difference between the first longitudinal coordinates of the two diagonal endpoints of the target slot and the second longitudinal coordinates of the two endpoints of the first sub-slot is calculated to obtain the first coordinate difference corresponding to the two endpoints respectively; when both first coordinate differences are less than or equal to the slot deviation threshold in the tolerance parameter, the second target state of the target slot is marked as the slot normal state, and the slot marking information corresponding to the first sub-slot is marked as the slot matched state, and the search for the current target slot ends, and the next target slot is processed. After traversing all the first sub-slots in the first slot information, if there is no first sub-slot whose two first coordinate differences with the target slot are both less than or equal to the slot deviation threshold, then the second target state of the target slot is marked as a slot abnormal state, and the next target slot is processed. When the second slot direction is the longitudinal slot, the second sub-slots in the second slot information are traversed, and the difference between the first lateral coordinates of the two diagonal endpoints of the target slot and the second lateral coordinates of the two endpoints of the second sub-slot is calculated to obtain the second coordinate difference values ​​corresponding to the two endpoints respectively; when both second coordinate differences are less than or equal to the slot deviation threshold, the second target state of the target slot is marked as the normal state of the slot, and the slot marking information corresponding to the second sub-slot is marked as the matched state of the slot, and the search for the current target slot ends, and the next target slot is processed. After traversing all the second sub-slots in the second slot information, if there is no second sub-slot whose two second coordinate differences with the target slot are both less than or equal to the slot deviation threshold, then the second target state of the target slot is marked as the slot abnormal state, and the next target slot is processed. Repeat the above steps until the second target state of all the target slots is obtained, wherein when each target slot is matched with the first sub-slot or the second sub-slot, the design slots whose slot marking information has been marked as the slot matched state are skipped; Data processing is performed on all the target slots in the second target state to obtain the information matching result corresponding to the target slot in the current posture of the first candidate drawing.

7. The method according to claim 6, characterized in that, The step of obtaining target defect information corresponding to the target board material by analyzing the results and determining defects based on the target matching results and the candidate chipping regions includes: The target drawing is determined based on the drawing number and the drawing posture, and the number of valid measured holes, the number of matching holes, and the number of unmatched holes are determined based on the matching index and the target drawing. If the number of valid measured holes is greater than the number of holes that can be matched in the drawing, the hole defect type corresponding to the target board is determined to be a porous defect; if the number of valid measured holes is less than the number of holes that can be matched in the drawing, the hole defect type is determined to be a leaky hole defect; if the number of valid measured holes is equal to the number of holes that can be matched in the drawing but the number of unmatched holes is greater than a preset number, the hole defect type is determined to be a hole position deviation; and the relevant geometric features corresponding to the leaky hole information under the number of unmatched holes are compared with the chipping candidate area. If the relevant distance between the relevant geometric features and the chipping candidate area is less than or equal to a preset distance, then the... The first target state corresponding to the hole information is determined as the normal state of the hole; if the relevant distance is greater than the preset distance, then the first local grayscale information is obtained from the candidate image according to the relevant geometric features, and the second local grayscale information corresponding to the background around the relevant geometric features is obtained from the candidate image. The brightness difference between the first local grayscale information and the second local grayscale information is compared. When the brightness difference is greater than the preset brightness, the first target state corresponding to the hole information is determined as the normal state of the hole; otherwise, the first target state corresponding to the hole information is determined as the abnormal state of the hole. Based on the matching criteria and the target drawing, determine the number of valid measured slots, the number of slots that can be matched with the drawing, and the number of unmatched slots. When the number of valid measured slots is greater than the number of slots that can be matched on the drawing, the slot defect type corresponding to the target board is determined to be a multi-slot defect; when the number of valid measured slots is less than the number of slots that can be matched on the drawing, and there are design slots whose slot marking information is not marked as matching, the slot defect type is determined to be a missing slot defect; when the number of valid measured slots is equal to the number of slots that can be matched on the drawing, but the endpoint coordinate deviation between the target slot and the design slot exceeds a set threshold, the slot defect type is determined to be a slot misalignment defect. The target defect information corresponding to the target plate is determined based on the groove defect type, the hole defect type, the first target state corresponding to the target hole, and the second target state corresponding to the target groove.

8. A flexible detection system for hole and groove defects in finished sheet metal, characterized in that, The system includes: The region cropping module is used to crop the candidate region based on the effective range of the target board in the board image to obtain the candidate image; The image processing module is used to preprocess and process the edge contour of the candidate image to obtain the corresponding edge image, and to filter and analyze the candidate image based on the edge image to obtain the corresponding target candidate region in the candidate image. The target candidate region includes the measured hole region, the chipping candidate region, and the slot candidate region. The drawing filtering module is used to attempt to filter the board image using both the front and back sides to obtain the corresponding valid drawings, and to determine the corresponding tolerance parameters according to the board type corresponding to the target board. The coordinate transformation module is used to determine the drawing coordinate system corresponding to the valid drawing and the image coordinate system corresponding to the candidate image, and to establish an affine matrix from the candidate image to the valid drawing based on the drawing coordinate system and the image coordinate system. The feature processing module is used to perform geometric analysis on the measured hole area and the slot candidate area to obtain initial geometric features, and to perform information transformation on the initial geometric features according to the affine matrix to obtain the target geometric features corresponding to the drawing coordinate system. The information matching module is used to perform attitude traversal based on the valid drawings and the target geometric features, and combine the tolerance parameters to match the measured holes and slots with the designed holes and slots in the valid drawings to obtain the target matching result; The defect determination module is used to perform result analysis and defect determination based on the target matching result and the chipping candidate area to obtain the target defect information corresponding to the target board.

9. A terminal device, characterized in that, The terminal device includes a processor and a memory; The memory is used to store computer programs; The processor is used to execute the computer program and, when executing the computer program, implement the flexible detection method for hole and groove defects in finished sheet metal as described in any one of claims 1 to 7.

10. A computer storage medium for computer storage, characterized in that, The computer storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the flexible detection method for hole and groove defects in finished sheet metal according to any one of claims 1 to 7.