Defect detection method and device, equipment and storage medium

Abstract

The invention discloses a defect detection method, a defect detection device, defect detection equipment and a storage medium, and relates to the technical field of image processing. According to the position information of the positioning template area of the preset template image, positioning points are matched in the to-be-detected image, and the coordinates of the positioning points are obtained; the coordinate transformation relation between the preset template image and the to-be-detected image can be known by combining the positioning point coordinates and the position information of the positioning template area, and then the position of the detection item area in the preset template image is accurately mapped to the to-be-detected image to obtain a detection item mapping area; furthermore, the correlation between the target detection item area and the detection item mapping area is compared to obtain a defect detection result, manual detection is not needed, and the detection accuracy and efficiency are effectively improved.

Description

Defect detection methods, devices, equipment and storage media Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a defect detection method, apparatus, device, and storage medium. Background Technology

[0002] Currently, the appearance inspection of industrial control computers still largely relies on manual inspection. This inspection involves several key aspects, including the installation status of screws, the physical form of various interfaces (such as power connectors, network ports, and USB ports), and the affixing and integrity of product labels. Quality inspectors must visually inspect each industrial control computer part by part on the assembly line or before the inspection station. As working hours increase, fatigue gradually accumulates, directly leading to a decline in inspection focus, and consequently, lower inspection accuracy and efficiency.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this application is to provide a defect detection method, apparatus, equipment, and storage medium, which aims to improve the accuracy and efficiency of defect detection in industrial control computers.

[0005] To achieve the above objectives, this application proposes a defect detection method, the method comprising: acquiring an image to be detected; performing positioning point matching in the image to be detected based on the position information of at least one positioning template region in a preset template image to obtain the positioning point coordinates of at least one target positioning point; mapping the position information of a target detection item region in the preset template image to the image to be detected based on the positioning point coordinates of each target positioning point and the position information of each positioning template region to obtain a detection item mapping region, wherein the target detection item region refers to the region of the item to be detected; and performing correlation detection on the target detection item region and the detection item mapping region to obtain a defect detection result.

[0006] In one embodiment, the position information of the positioning template region includes the coordinates of the center point of the positioning template region; each positioning template region corresponds to a target positioning point; the step of mapping the position information of the target detection item region in the preset template image to the image to be detected based on the positioning point coordinates of each target positioning point and the position information of each positioning template region to obtain the detection item mapping region includes: determining the distance between the center point coordinates of any two positioning template regions, and determining the distance between the positioning point coordinates of any two target positioning points; selecting reference point coordinates from the positioning point coordinates of each target positioning point based on the distance between the center point coordinates of any two positioning template regions and the distance between the positioning point coordinates of any two target positioning points; constructing a transformation matrix based on the reference point coordinates and the center point coordinates of the positioning template region associated with the reference point coordinates; and mapping the position information of the target detection item region to the image to be detected based on the transformation matrix to obtain the detection item mapping region.

[0007] In one embodiment, the step of selecting reference point coordinates from the positioning point coordinates of each target positioning point based on the distance between the center point coordinates of any two positioning template areas and the distance between the positioning point coordinates of any two target positioning points includes: for each pair of positioning template areas, calculating the absolute value of the difference based on the distance between the center point coordinates of the two positioning template areas and the distance between the positioning point coordinates of the two target positioning points, wherein the two target positioning points correspond one-to-one with the two positioning template areas; comparing the absolute value of each difference with a preset difference threshold; and selecting the reference point coordinates from the positioning point coordinates of each target positioning point based on the difference comparison result.

[0008] In one embodiment, the step of constructing a transformation matrix based on the reference point coordinates and the center point coordinates of the positioning template area associated with the reference point coordinates includes: substituting the reference point coordinates and the center point coordinates of the positioning template area associated with the reference point coordinates into a preset coordinate relationship matrix formula, solving for the equation coefficients in the coordinate relationship matrix formula; and constructing the transformation matrix based on the equation coefficients.

[0009] In one embodiment, the step of mapping the position information of the target detection item region to the image to be detected according to the transformation matrix to obtain the detection item mapping region includes: multiplying the position information of the target detection item region with the transformation matrix to obtain mapping position information; and determining the detection item mapping region in the image to be detected according to the mapping position information.

[0010] In one embodiment, the step of matching positioning points in the image to be detected based on the position information of at least one positioning template region in a preset template image to obtain the positioning point coordinates of at least one target positioning point includes: for each positioning template region, performing the following operations: extracting the first grayscale value of each pixel in the preset template image based on the position information of the positioning template region; performing a sliding traversal in the image to be detected to extract the second grayscale value of each pixel from the currently traversed target region; determining the pixel similarity based on each first grayscale value and each second grayscale value; if the pixel similarity exceeds a preset similarity threshold, then using the coordinates of the center point of the target region as the positioning point coordinates of the target positioning point.

[0011] In one embodiment, the step of performing correlation detection on the target detection item region and the detection item mapping region to obtain a defect detection result includes: extracting a third gray value corresponding to each pixel in the target detection item region; extracting a fourth gray value corresponding to each pixel in the detection item mapping region; determining a linear correlation coefficient based on each of the third gray values ​​and each of the fourth gray values; and comparing the linear correlation coefficient with a preset correlation threshold to determine the defect detection result based on the comparison result.

[0012] Furthermore, to achieve the above objectives, this application also proposes a defect detection device, comprising: an acquisition module for acquiring an image to be detected; a positioning point matching module for performing positioning point matching in the image to be detected based on the position information of at least one positioning template region in a preset template image to obtain the positioning point coordinates of at least one target positioning point, wherein each positioning template region corresponds to a target positioning point; a mapping module for mapping the position information of a target detection item region in the preset template image to the image to be detected based on the positioning point coordinates of each target positioning point and the position information of each positioning template region to obtain a detection item mapping region, wherein the target detection item region refers to the region of the item to be detected; and a detection module for performing correlation detection on the target detection item region in the preset template image and the detection item mapping region in the image to be detected to obtain a defect detection result.

[0013] In addition, to achieve the above objectives, this application also proposes a defect detection device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the defect detection method as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the defect detection method described above.

[0015] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the defect detection method described above.

[0016] This application provides a defect detection method, apparatus, device, and storage medium. The defect detection method acquires an image to be detected; based on the position information of the positioning template region of a preset template image, it matches positioning points in the image to be detected and obtains the coordinates of the positioning points; by combining the coordinates of the positioning points with the position information of the positioning template region, it can understand the coordinate transformation relationship between the preset template image and the image to be detected, and then accurately maps the position of the detection item region in the preset template image to the image to be detected, thus obtaining the detection item mapping region; furthermore, by comparing the correlation between the target detection item region and the detection item mapping region, the defect detection result is obtained, eliminating the need for manual inspection and effectively improving the accuracy and efficiency of detection. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 is a flowchart illustrating the defect detection method of this application in Embodiment 1; Figure 2 is a flowchart illustrating the defect detection method of this application in Embodiment 2; Figure 3 is a schematic diagram of a preset template image provided in Embodiment 1 of this application; Figure 4 is a schematic diagram of an image to be detected provided in Embodiment 1 of this application; Figure 5 is a flowchart illustrating the defect detection method of this application in Embodiment 3; Figure 6 is a flowchart illustrating the defect detection method of this application in Embodiment 4; Figure 7 is a detailed flowchart illustrating the defect detection process provided in Embodiment 1 of this application; Figure 8 is a schematic diagram of the module structure of the defect detection device in an embodiment of this application; Figure 9 is a schematic diagram of the device structure of the hardware operating environment involved in the defect detection method in an embodiment of this application.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device, big data service platform, or defect detection system capable of performing the above functions. The following description uses a defect detection system as an example to illustrate this embodiment and the subsequent embodiments.

[0024] It should be noted that the defect detection method of this application embodiment can be used for various types of equipment that need to perform appearance defect detection. In addition to industrial control computers, it can also be applied to other equipment in the field of industrial control, consumer electronic devices, vehicle-related equipment, security monitoring equipment, medical auxiliary equipment, etc. This embodiment takes an industrial control computer as an example for illustration.

[0025] This application provides a defect detection method. Referring to Figure 1, Figure 1 is a flowchart of the defect detection method provided in Embodiment 1 of this application.

[0026] Step S11: Obtain the image to be detected. It should be noted that the image to be detected can be an image of the industrial control computer acquired by an image acquisition device.

[0027] Step S12: Based on the position information of at least one positioning template region in the preset template image, perform positioning point matching in the image to be detected to obtain the positioning point coordinates of at least one target positioning point.

[0028] It should be noted that the item to be detected in the industrial control computer is placed perpendicular to the camera so that the image acquisition device can acquire a preset template image. The positioning template area is a pre-set area in the template image. Optionally, the positioning template area is selected in a high-contrast area of ​​the image. In addition, there is no limit to the number of positioning template areas, for example, 3. Optionally, the positioning template areas are not on the same straight line.

[0029] It should be noted that the location information of the positioning template region includes the coordinates of the center point of the positioning template region. Each positioning template region corresponds to a target positioning point, that is, a target positioning point is matched in the image to be detected for each positioning template region.

[0030] For each positioning template region, the following operations are performed: In one embodiment, based on the position information of at least one positioning template region in the preset template image, a target region with a high similarity to the positioning template region is obtained by traversing the image to be detected. Optionally, based on the gray value of each pixel in the positioning template region, a target region with a high similarity to the gray value of the positioning template region is obtained by traversing the image to be detected.

[0031] In other feasible embodiments, edge detection can also be performed on the positioning template region of the preset template image. For example, edge detection can be performed using algorithms such as Canny and Sobel to obtain the edge features of the positioning template region. Candidate regions are obtained by traversing the image to be detected, and the edge features of the candidate regions are extracted. During the traversal, the similarity between the edge features of the positioning template region and the edge features of the candidate regions is compared. If the similarity meets the standard, it is the target region.

[0032] In other feasible embodiments, texture detection can be performed on the positioning template region of the preset template image to obtain the texture features of the positioning template region; candidate regions are obtained by traversing the image to be detected, and the texture features of the candidate regions are extracted. During the traversal, the similarity between the texture features of the positioning template region and the texture features of the candidate regions is compared. If the similarity meets the standard, it is the target region.

[0033] In other embodiments, to improve the accuracy of location point matching, the similarity of grayscale values, edge features, and texture features can be combined to determine the target region.

[0034] It should be noted that the computational load of template matching can be reduced by setting the angle search range during template matching, thereby shortening the total time spent on template matching. Optionally, the starting angle and range length of the search can be preset. For example, if the starting angle is set to -5° and the range length is set to 10°, then the search range is [-5°, 5°]. That is, in the image to be detected, the positioning points are matched according to the rotation angle in the range of [-5°, 5°].

[0035] Furthermore, the center point coordinates of the center point of the target area are determined, and then the center point coordinates of the center point of the target area are used as the positioning point coordinates of the target positioning point.

[0036] Step S13: Based on the coordinates of each target positioning point and the position information of each positioning template area, map the position information of the target detection item area in the preset template image to the image to be detected, thus obtaining the detection item mapping area. It should be noted that the target detection item area refers to the area of ​​the item to be detected. Optionally, the item to be detected includes screws, interfaces, and labels on the exterior of an industrial control computer. For example, it checks whether the screws are missing or misaligned, whether various interfaces (e.g., power interfaces, network ports, USB interfaces, etc.) are deformed, damaged, or have bent pins, and whether product labels are misaligned, wrinkled, have incorrect information printing, or are detached or damaged. The target detection item area is usually selected using a rectangle to frame the detection items (screws, interfaces, labels, etc.).

[0037] It should be noted that the location information of the target detection area includes the coordinates of the top-left and bottom-right corners of the target detection area, and / or the coordinates of the bottom-left and top-right corners. The pixel size of the target detection area is the same as that of the detection mapping area.

[0038] In this embodiment, a transformation matrix between the preset template image and the image to be detected is constructed based on the coordinates of each target location point in the image to be detected and the position information of each location template region in the preset template image. This transformation matrix can be used to characterize the coordinate transformation relationship between the preset template image and the image to be detected. Then, using the transformation matrix, the position information of the target detection item region is mapped onto the image to be detected to obtain the detection item mapping region.

[0039] Step S14: Perform correlation detection on the target detection item region and the detection item mapping region to obtain the defect detection result.

[0040] It should be noted that correlation detection refers to detecting the correlation between the gray values ​​of pixels in the target detection area and the detection item mapping area. In other embodiments, the correlation between the texture features and edge features of pixels in the target detection area and the detection item mapping area can also be detected.

[0041] In this embodiment, the correlation between the grayscale values, texture features, and / or edge features of pixels in the target detection area and the detection mapping area is calculated. The correlation is compared with a preset correlation threshold. If the correlation is greater than the preset correlation threshold, the defect detection result of the image to be detected is determined to be without defects. If the correlation is less than or equal to the preset correlation threshold, the defect detection result of the image to be detected is determined to be defective.

[0042] This embodiment acquires the image to be detected; based on the position information of the positioning template area of ​​the preset template image, it matches positioning points in the image to be detected and obtains the coordinates of the positioning points; by combining the coordinates of the positioning points with the position information of the positioning template area, it can understand the coordinate transformation relationship between the preset template image and the image to be detected, and then accurately maps the reference position of the detection item in the preset template image to the image to be detected, thus obtaining the detection item mapping area; furthermore, by comparing the correlation between the target detection item area and the detection item mapping area, the defect detection result is obtained, eliminating the need for manual inspection and effectively improving the detection accuracy and efficiency.

[0043] In one feasible implementation, referring to Figure 2, which is a flowchart of the defect detection method according to Embodiment 2 of this application; the location information of the positioning template area includes the coordinates of the center point of the positioning template area; each positioning template area corresponds to a target positioning point; the step of mapping the location information of the target detection item area in the preset template image to the image to be detected according to the positioning point coordinates of each target positioning point and the location information of each positioning template area to obtain the detection item mapping area includes: step S21, determining the distance between the center point coordinates of any two positioning template areas, and determining the distance between the positioning point coordinates of any two target positioning points; it should be noted that the number of positioning template areas can be set according to the actual situation. For example, referring to Figure 3, which is a schematic diagram of a preset template image provided in an embodiment of this application, the number of positioning template areas is set to 3. The positioning template area includes a first positioning template area (positioning template 1 in Figure 3), a second positioning template area (positioning template 2 in Figure 3), and a third positioning template area (positioning template 3 in Figure 3).

[0044] Referring to Figure 4, which is a schematic diagram of an image to be detected according to an embodiment of this application, the target positioning points include a first target positioning point (template matching positioning point 1 in Figure 4), a second target positioning point (template matching positioning point 2 in Figure 4) and a third target positioning point (template matching positioning point 3 in Figure 4).

[0045] It should be noted that each positioning template area corresponds to a target positioning point. The first target positioning point is obtained by matching the first positioning template area, the second target positioning point is obtained by matching the second positioning template area, and the third target positioning point is obtained by matching the third positioning template area.

[0046] In this embodiment, the distance between the center point coordinates of any two positioning template areas is calculated, including the first distance between the center point coordinates of the first positioning template area and the center point coordinates of the second positioning template area (L12 in Figure 3), the second distance between the center point coordinates of the second positioning template area and the center point coordinates of the third positioning template area (L23 in Figure 3), and the third distance between the center point coordinates of the first positioning template area and the center point coordinates of the third positioning template area (L13 in Figure 3).

[0047] In addition, the distance between the coordinates of any two target positioning points is calculated, including the fourth distance between the coordinates of the first target positioning point and the coordinates of the second target positioning point (L12' in Figure 4), the fifth distance between the coordinates of the second target positioning point and the coordinates of the third target positioning point (L23' in Figure 4), and the sixth distance between the coordinates of the first target positioning point and the coordinates of the third target positioning point (L13' in Figure 4).

[0048] It should be noted that if the coordinates of point A are (x1, y1) and the coordinates of point B are (x2, y2), the distance is calculated as shown in formula (1): (1) Where dist represents distance.

[0049] Step S22: Based on the distance between the center point coordinates of any two positioning template areas and the distance between the positioning point coordinates of any two target positioning points, a reference point coordinate is selected from the positioning point coordinates of each target positioning point. In this embodiment, for each pair of positioning template areas, which includes any two positioning template areas, the following operations are performed: Determine the two target positioning points corresponding to the two positioning template areas in the pair. For example, the first positioning template area corresponds to the first target positioning point, the second positioning template area corresponds to the second target positioning point, and the third positioning template area corresponds to the third target positioning point. Further, based on the distance between the center point coordinates of the two positioning template areas and the distance between the positioning point coordinates of the two target positioning points, the absolute value of the difference is calculated.

[0050] Furthermore, the absolute value of the difference between each pair of positioning template areas is compared with a preset difference threshold, where the preset difference threshold can be set according to actual conditions and is not specifically limited here. Based on the difference comparison results between each pair of positioning template areas, the coordinates of the reference point are selected from the coordinates of the first target positioning point, the coordinates of the second target positioning point, and the coordinates of the third target positioning point.

[0051] In a feasible implementation manner, step S22 includes: step S221, for every two of the positioning template regions, calculate the absolute value of the difference according to the distance between the center point coordinates of the two positioning template regions and the distance between the positioning point coordinates of the two target positioning points, where the two target positioning points correspond to the two positioning template regions respectively; step S222, compare the absolute values of the differences with a preset difference threshold; step S223, select the positioning point coordinates of the reference point from the positioning point coordinates of each of the target positioning points according to the difference comparison result.

[0052] Optionally, determine the absolute value of the difference between the first distance and the fourth distance, that is, |L12 - L12'|. Determine the absolute value of the difference between the second distance and the fifth distance, that is, |L23 - L23'|. Determine the absolute value of the difference between the third distance and the sixth distance, that is, |L13 - L13'|.

[0053] Further, judge whether |L12 - L12'| is less than D, whether |L23 - L23'| is less than D, and whether |L13 - L13'| is less than D, where D represents the preset difference threshold.

[0054] If |L12 - L12'| < D and |L23 - L23'| < D and |L13 - L13'| < D, that is, the absolute values of all differences are less than the preset difference threshold, then select the positioning point coordinates of the first target positioning point, the positioning point coordinates of the second target positioning point, and the positioning point coordinates of the third target positioning point together as the reference point coordinates of the reference point, that is, jointly combine the positioning point coordinates of the first target positioning point, the positioning point coordinates of the second target positioning point, and the positioning point coordinates of the third target positioning point to determine the detected item area after mapping.

[0055] If |L12 - L12'| > D, |L23 - L23'| < D, and |L13 - L13'| > D, then select the positioning point coordinates of the second target positioning point and the positioning point coordinates of the third target positioning point together as the reference point coordinates of the reference point, that is, jointly combine the positioning point coordinates of the second target positioning point and the positioning point coordinates of the third target positioning point to determine the detected item area after mapping.

[0056] If |L12 - L12'| > D, |L23 - L23'| > D, and |L13 - L13'| < D, then select the positioning point coordinates of the first target positioning point and the positioning point coordinates of the third target positioning point together as the reference point coordinates of the reference point, that is, jointly combine the positioning point coordinates of the first target positioning point and the positioning point coordinates of the third target positioning point to determine the detected item area after mapping.

[0057] If |L12 - L12'| < D, |L23 - L23'| > D, and |L13 - L13'| > D, then select the positioning point coordinates of the first target positioning point and the positioning point coordinates of the second target positioning point together as the reference point coordinates of the reference point. That is, jointly combine the positioning point coordinates of the first target positioning point and the positioning point coordinates of the second target positioning point to determine the detected item area after mapping.

[0058] It can be understood that if only the absolute value of one difference is less than the preset difference threshold, then determine the two target positioning points corresponding to the absolute value less than the preset difference threshold, and then jointly use the two target positioning points as the reference point coordinates of the reference point.

[0059] If |L12 - L12'| > D, |L23 - L23'| > D, and |L13 - L13'| > D, then select the positioning point coordinates of the first target positioning point, the positioning point coordinates of the second target positioning point, and the positioning point coordinates of the third target positioning point respectively as the reference point coordinates of the reference point.

[0060] Step S23, construct a transformation matrix according to the reference point coordinates and the center point coordinates of the positioning template area associated with the reference point coordinates; in this embodiment, substitute the reference point coordinates and the center point coordinates of the positioning template area associated with the reference point coordinates into a preset coordinate relationship matrix formula to solve for the equation coefficients in the coordinate relationship matrix formula; the coordinate relationship matrix formula includes an X-axis coordinate relationship matrix formula and a Y-axis coordinate relationship matrix formula. Then, construct the transformation matrix according to the equation coefficients.

[0061] More specifically, if |L12 - L12'| < D and |L23 - L23'| < D and |L13 - L13'| < D, then substitute the positioning point coordinates of the first target positioning point, the second target positioning point, and the third target positioning point, as well as the first positioning template area, the second positioning template area, and the third positioning template area into a preset coordinate relationship matrix formula, and use the least squares method to solve for the equation coefficients in the coordinate relationship matrix formula. Then, construct a transformation matrix according to the equation coefficients. Among them, the transformation matrix is a 3x3 homogeneous transformation matrix, and the reasoning process of the transformation matrix is as follows: First, assume that the center point coordinates of the known positioning template area are: OriginalX (X-axis coordinate) = [W1, W2, W3], OriginalY (Y-axis coordinate) = [U1, U2, U3], and the reference point coordinates are: TransformedX (X-axis coordinate) = [W1', W2', W3'], TransformedY (Y-axis coordinate) = [U1', U2', U3']. Among them, for the X-axis coordinate, there is a matrix relationship as shown in formula (2): (2) There is a matrix relationship for the Y-axis coordinate as shown in formula (3): (3) Among them, a, b, c, d, e, f are the unknown equation coefficients.

[0062] Then, through formula (2) and formula (3), the specific values of the equation coefficients a, b, c, d, e, f are calculated using the least squares method.

[0063] Then, according to the specific values of the equation coefficients a, b, c, d, e, f, a transformation matrix M is constructed. Among them, the transformation matrix M is as shown in formula (4): (4) In one embodiment, when |L12 - L12'| > D, |L23 - L23'| < D, and |L13 - L13'| > D, or when |L12 - L12'| > D, |L23 - L23'| > D, and |L13 - L13'| < D, or when |L12 - L12'| < D, |L23 - L23'| > D, and |L13 - L13'| > D, according to the reference point coordinates and the center point coordinates of the positioning template area associated with the reference point coordinates, the transformation matrix is constructed. The transformation matrix is a 2x2 homogeneous transformation matrix. Among them, the reasoning process of the transformation matrix is similar to the above reasoning process and will not be elaborated here.

[0064] In one embodiment, when |L12 - L12'| > D, |L23 - L23'| > D, and |L13 - L13'| > D: The positioning point coordinates of the first target positioning point, the positioning point coordinates of the second target positioning point, and the positioning point coordinates of the third target positioning point are respectively selected as the reference point coordinates of the reference point. Therefore, it is necessary to construct a transformation matrix according to the positioning point coordinates of each target positioning point and the center point coordinates of the positioning template area associated with the target positioning point. For example, determine the angle of the first positioning template area and the angle of the first target positioning point, and then construct a transformation matrix according to the angle, the positioning point coordinates of the first target positioning point, and the center point coordinates of the first positioning template area. Among them, the reasoning process of the transformation matrix is similar to the above reasoning process and will not be elaborated here.

[0065] Step S24, according to the transformation matrix, map the position information of the target detection item area to the to-be-detected image to obtain the detection item mapping area.

[0066] It should be noted that the position information of the target detection item area includes the upper left corner coordinates and the lower right corner coordinates of the target detection item area, and / or the lower left corner coordinates and the upper right corner coordinates.

[0067] In this embodiment, the transformation matrix refers to the coordinate transformation relationship between the preset template image and the image to be detected. Therefore, by using the transformation matrix to map the position information of the target detection item region, the position of the target detection item region mapped to the image to be detected can be calculated, thereby obtaining the detection item mapping region.

[0068] In one feasible implementation, step S24 includes: step S241, multiplying the position information of the target detection item region with the transformation matrix to obtain mapping position information; step S242, determining the detection item mapping region in the image to be detected based on the mapping position information.

[0069] In this embodiment, the transformation matrix is ​​multiplied by the coordinates of the upper left corner and the lower right corner of the target detection item region, and / or the coordinates of the lower left corner and the upper right corner, to obtain the mapping position information mapped to the image to be detected. The mapping position information includes the mapped upper left corner and the lower right corner, and / or the coordinates of the lower left corner and the upper right corner. Further, the area formed by the mapping position information in the image to be detected is used as the detection item mapping region.

[0070] It should be noted that in the cases of |L12-L12'|>D, |L23-L23'|>D, and |L13-L13'|>D, multiple transformation matrices will be constructed. The position information of the target detection item region needs to be multiplied by each of the transformation matrices to obtain multiple detection item mapping regions.

[0071] This embodiment obtains reference point coordinates from the coordinates of the target positioning points by selecting them from the coordinates of the positioning points of each target positioning point based on the distance between the center point coordinates of any two positioning template regions and the distance between the coordinates of the positioning points of any two target positioning points. A transformation matrix is ​​constructed based on the reference point coordinates and the center point coordinates of the positioning template regions associated with the reference point coordinates. Then, the position information of the target detection item region is mapped onto the image to be detected based on the transformation matrix, resulting in the detection item mapping region. Mapping the detection item mapping region using multiple positioning templates provides high robustness; even if a positioning template is missing or occluded, target positioning can still be achieved. This reduces the probability of false detections caused by the failure of a single template positioning and also reduces anomalies in the detection item mapping position caused by image deformation, effectively improving the accuracy of defect detection.

[0072] In one feasible implementation, referring to Figure 5, which is a flowchart of the defect detection method according to Embodiment 3 of this application; based on the position information of at least one positioning template region in a preset template image, positioning point matching is performed in the image to be detected to obtain the positioning point coordinates of at least one target positioning point, including: for each positioning template region, the following operations are performed: Step S31, extracting the first gray value of each pixel in the preset template image based on the position information of the positioning template region; Step S32, performing a sliding traversal in the image to be detected to extract the second gray value of each pixel from the currently traversed target region; Step S33, determining the pixel similarity based on each first gray value and each second gray value; Step S34, if the pixel similarity exceeds a preset similarity threshold, then the coordinates of the center point of the target region are used as the positioning point coordinates of the target positioning point.

[0073] It should be noted that the pixel size of the target detection area and the pixel position of the detection mapping area are consistent, and they correspond one-to-one. In this embodiment, the first gray value of each pixel in the positioning template area is extracted according to the position information of the positioning template area; the image to be detected is traversed according to a preset sliding time window, wherein the size of the sliding time window is the same as the pixel size of the positioning template area. Optionally, all areas in the image to be detected are traversed without omission in the order from left to right and from top to bottom. In addition, during the traversal, the second gray value of each pixel in the currently traversed target area needs to be extracted. Further, the pixel similarity is calculated based on each first gray value and each second gray value; wherein, the pixel similarity is the linear correlation coefficient of gray values, used to measure the consistency of the changing trends of two gray value arrays. In other embodiments, the pixel similarity can also be the normalized cross-correlation (NCC) of gray values. Furthermore, if the similarity of the target region exceeds the preset similarity threshold, it means that the target region matches the positioning template region, and the center point coordinates of the target region are used as the positioning point coordinates of the target positioning point.

[0074] This embodiment first extracts grayscale values ​​from the positioning template region of the template image, and then uses a sliding window of the same size as the positioning template region to traverse the image to be detected. Based on the target region obtained in each traversal, the second grayscale value of each pixel is extracted, and then the similarity between the grayscale values ​​of the two is calculated. If the similarity exceeds a preset similarity threshold, the coordinates of the center point of the target region are used as the positioning point coordinates of the target positioning point. Subsequently, based on the positioning point coordinates of each target positioning point and the position information of each positioning template region, the coordinate transformation relationship between the preset template image and the image to be detected can be constructed, thereby accurately mapping the detection item mapping region and effectively improving the detection accuracy.

[0075] In one feasible implementation, referring to Figure 6, which is a flowchart of the defect detection method according to Embodiment 4 of this application; the correlation detection of the target detection item region and the detection item mapping region is performed to obtain the defect detection result, including: step S41, extracting the third gray value corresponding to each pixel in the target detection item region; step S42, extracting the fourth gray value corresponding to each pixel in the detection item mapping region; step S43, determining the linear correlation coefficient based on each of the third gray value and each of the fourth gray value; step S44, comparing the linear correlation coefficient with a preset correlation threshold to determine the defect detection result based on the comparison result.

[0076] It should be noted that the pixel size and pixel position of the target detection area and the detection item mapping area are consistent. For example, the (1,1) pixel in the target detection area corresponds to the (1,1) pixel in the detection item mapping area. The linear correlation coefficient is the consistency of the trend of grayscale change.

[0077] In this embodiment, in the target detection area, the third grayscale value corresponding to each pixel is extracted point by point, and in the detection item mapping area, the fourth grayscale value corresponding to each pixel is extracted point by point. The order of grayscale value extraction is the same in both cases; for example, the target detection area extracts grayscale values ​​from left to right and from top to bottom, and the detection item mapping area extracts grayscale values ​​in the same order. Further, a linear correlation coefficient is calculated based on each of the third and fourth grayscale values. The calculation process for the linear correlation coefficient is a well-established existing technology and will not be elaborated here. Further, the linear correlation coefficient is compared with a preset correlation threshold. The preset correlation threshold can be set according to actual conditions and is not specifically limited here. If the linear correlation coefficient is greater than the preset correlation threshold, it indicates that the grayscale trend is consistent, and the defect detection result of the image to be detected is determined to be defect-free. If the linear correlation coefficient is less than or equal to the preset correlation threshold, it indicates that the defect detection result of the image to be detected is defective, such as scratches, deformation, or missing parts causing grayscale abnormalities.

[0078] Furthermore, it should be noted that in the cases where |L12-L12'|>D, |L23-L23'|>D, and |L13-L13'|>D: Since multiple detection item mapping regions are mapped in the image to be detected, the aforementioned correlation detection process needs to be performed for each of these detection item mapping regions to obtain the linear correlation coefficient corresponding to each region. Each linear correlation coefficient is compared with a preset correlation threshold. If any linear correlation coefficient is greater than the preset correlation threshold, the defect detection result of the image to be detected is determined to be defect-free. If all linear correlation coefficients are less than or equal to the preset correlation threshold, the defect detection result of the image to be detected indicates the presence of a defect.

[0079] This embodiment extracts the grayscale values ​​of each pixel in the target detection area and the detection mapping area respectively, and calculates the linear correlation coefficient based on the grayscale values ​​of each pixel in the two areas; then, the defect detection result is determined based on the linear correlation coefficient, eliminating the need for manual inspection and effectively improving detection accuracy and efficiency.

[0080] Referring to FIG. 7, FIG. 7 is a schematic diagram of the specific process of defect detection provided by an embodiment of the present application; specifically, according to a preset template image, 3 positioning template regions are created, and a target detection item region is set. The 3 positioning template regions are used to perform positioning point matching in the image to be detected, and the positioning point coordinates of 3 target positioning points are obtained. The distances (L12, L23, L13) between the center point coordinates of any two positioning template regions are determined, and the distances (L12', L23', L13') between the positioning point coordinates of any two target positioning points are determined; if |L12 - L12'| < D and |L23 - L23'| < D and |L13 - L13'| < D, that is, the absolute values of all differences are less than a preset difference threshold, then the positioning point coordinates of the first target positioning point, the positioning point coordinates of the second target positioning point, and the positioning point coordinates of the third target positioning point are jointly used as reference points; if |L12 - L12'| > D, |L23 - L23'| < D, and |L13 - L13'| > D, then the positioning point coordinates of the second target positioning point and the positioning point coordinates of the third target positioning point are jointly used as the reference point coordinates of the reference point; if |L12 - L12'| > D, |L23 - L23'| > D, and |L13 - L13'| < D, then the positioning point coordinates of the first target positioning point and the positioning point coordinates of the third target positioning point are jointly used as the reference point coordinates of the reference point; if |L12 - L12'| > D, |L23 - L23'| > D, and |L13 - L13'| > D, then the positioning point coordinates of the first target positioning point, the positioning point coordinates of the second target positioning point, and the positioning point coordinates of the third target positioning point are respectively used as the reference point coordinates of the reference point. According to the reference point coordinates, the position information of the target detection item region is mapped to the image to be detected, and a detection item mapping region is obtained. Among them, in the case of |L12 - L12'| > D, |L23 - L23'| > D, and |L13 - L13'| > D: multiple detection item mapping regions will be calculated. The target detection item region and the detection item mapping region are subjected to a correlation detection. If successful, it means there is no defect; if failed, it proves that there is a defect. And in the case of |L12 - L12'| > D, |L23 - L23'| > D, and |L13 - L13'| > D: if one of the detection item mapping regions is detected successfully, it means there is no defect.

[0081] It should be noted that the examples in the figure are only used to understand the present application and do not constitute a limitation on the defect detection method of the present application. Based on this technical concept, more forms of simple transformations are within the protection scope of the present application.

[0082] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The order of execution of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0083] This application also provides a defect detection device. Please refer to Figure 8, which is a schematic diagram of the module structure of the defect detection device according to an embodiment of this application. The defect detection device includes: an acquisition module 51, used to acquire an image to be detected; a positioning point matching module 52, used to perform positioning point matching in the image to be detected according to the position information of at least one positioning template region in a preset template image to obtain the positioning point coordinates of at least one target positioning point, wherein each positioning template region corresponds to a target positioning point; a mapping module 53, used to map the position information of the target detection item region in the preset template image to the image to be detected according to the positioning point coordinates of each target positioning point and the position information of each positioning template region to obtain a detection item mapping region, wherein the target detection item region refers to the region of the item to be detected; and a detection module 54, used to perform correlation detection on the target detection item region in the preset template image and the detection item mapping region in the image to be detected to obtain a defect detection result.

[0084] The defect detection device provided in this application, employing the defect detection method in the above embodiments, can solve the technical problems described in the background section. Compared with the prior art, the beneficial effects of the defect detection device provided in this application are the same as those of the defect detection method provided in the above embodiments, and other technical features in the defect detection device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0085] This application provides a defect detection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the defect detection method in Embodiment 1 above.

[0086] Referring to Figure 9, which is a schematic diagram of the hardware operating environment involved in the defect detection method in this embodiment of the application, the defect detection device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The defect detection device shown in Figure 9 is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0087] As shown in Figure 9, the defect detection device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the defect detection device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the defect detection equipment to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows defect detection equipment with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0088] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0089] The defect detection device provided in this application, employing the defect detection method in the above embodiments, can solve the technical problems described in the background section. Compared with the prior art, the beneficial effects of the defect detection device provided in this application are the same as those of the defect detection method provided in the above embodiments, and other technical features of the defect detection device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0090] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0091] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0092] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the defect detection method in the above embodiments.

[0093] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0094] The aforementioned computer-readable storage medium may be included in the defect detection equipment; or it may exist independently and not be assembled into the defect detection equipment.

[0095] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by a defect detection device, the defect detection device: acquires an image to be detected; matches a positioning point in the image to be detected and obtains the coordinates of the positioning point based on the position information of the positioning template region of a preset template image; combines the coordinates of the positioning point with the position information of the positioning template region to understand the coordinate transformation relationship between the preset template image and the image to be detected, thereby accurately mapping the position of the detection item region in the preset template image to the image to be detected, obtaining the detection item mapping region; furthermore, compares the correlation between the target detection item region and the detection item mapping region to obtain the defect detection result, eliminating the need for manual inspection and effectively improving detection accuracy and efficiency.

[0096] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0097] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0098] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0099] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described defect detection method, and is capable of solving the technical problems described in the background art. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the defect detection method provided in the above embodiments, and will not be repeated here.

[0100] This application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the defect detection method described above.

[0101] The computer program product provided in this application can solve the technical problems described in the background section. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiments of this application are the same as the beneficial effects of the defect detection method provided in the above embodiments, and will not be repeated here.

[0102] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A defect detection method, characterized in that, include: Acquire the image to be detected; Based on the position information of at least one positioning template region in the preset template image, positioning point matching is performed in the image to be detected to obtain the positioning point coordinates of at least one target positioning point. Based on the coordinates of each target positioning point and the position information of each positioning template area, the position information of the target detection item area in the preset template image is mapped to the image to be detected to obtain the detection item mapping area, wherein the target detection item area refers to the area of ​​the item to be detected; correlation detection is performed on the target detection item area and the detection item mapping area to obtain the defect detection result.

2. The defect detection method as described in claim 1, characterized in that, The location information of the positioning template region includes the coordinates of the center point of the positioning template region; each positioning template region corresponds to a target positioning point; the step of mapping the location information of the target detection item region in the preset template image to the image to be detected based on the positioning point coordinates of each target positioning point and the location information of each positioning template region to obtain the detection item mapping region includes: determining the distance between the center point coordinates of any two positioning template regions, and determining the distance between the positioning point coordinates of any two target positioning points; selecting reference point coordinates from the positioning point coordinates of each target positioning point based on the distance between the center point coordinates of any two positioning template regions and the distance between the positioning point coordinates of any two target positioning points; constructing a transformation matrix based on the reference point coordinates and the center point coordinates of the positioning template region associated with the reference point coordinates; and mapping the location information of the target detection item region to the image to be detected based on the transformation matrix to obtain the detection item mapping region.

3. The defect detection method as described in claim 2, characterized in that, The step of selecting reference point coordinates from the coordinates of each target positioning point based on the distance between the center point coordinates of any two positioning template areas and the distance between the coordinates of the positioning points of any two target positioning points includes: for each pair of positioning template areas, calculating the absolute value of the difference based on the distance between the center point coordinates of the two positioning template areas and the distance between the coordinates of the positioning points of the two target positioning points, wherein the two target positioning points correspond one-to-one with the two positioning template areas; comparing the absolute value of each difference with a preset difference threshold; and selecting the reference point coordinates from the coordinates of the positioning points of each target positioning point based on the difference comparison result.

4. The defect detection method as described in claim 2, characterized in that, The step of constructing a transformation matrix based on the coordinates of the reference point and the center point coordinates of the positioning template area associated with the reference point includes: substituting the coordinates of the reference point and the center point coordinates of the positioning template area associated with the reference point into a preset coordinate relationship matrix formula, and solving for the equation coefficients in the coordinate relationship matrix formula; and constructing the transformation matrix based on the equation coefficients.

5. The defect detection method as described in claim 2, characterized in that, The step of mapping the position information of the target detection item region to the image to be detected according to the transformation matrix to obtain the detection item mapping region includes: multiplying the position information of the target detection item region with the transformation matrix to obtain mapping position information; and determining the detection item mapping region in the image to be detected according to the mapping position information.

6. The defect detection method as described in claim 1, characterized in that, The step of matching positioning points in the image to be detected based on the position information of at least one positioning template region in a preset template image to obtain the positioning point coordinates of at least one target positioning point includes: for each positioning template region, performing the following operations: extracting the first grayscale value of each pixel in the preset template image based on the position information of the positioning template region; performing a sliding traversal in the image to be detected to extract the second grayscale value of each pixel from the currently traversed target region; determining the pixel similarity based on each first grayscale value and each second grayscale value; if the pixel similarity exceeds a preset similarity threshold, then using the coordinates of the center point of the target region as the positioning point coordinates of the target positioning point.

7. The defect detection method as described in claim 1, characterized in that, The step of performing correlation detection on the target detection item region and the detection item mapping region to obtain a defect detection result includes: extracting the third gray value corresponding to each pixel in the target detection item region; extracting the fourth gray value corresponding to each pixel in the detection item mapping region; determining a linear correlation coefficient based on each of the third gray value and each of the fourth gray value; and comparing the linear correlation coefficient with a preset correlation threshold to determine the defect detection result based on the comparison result.

8. A defect detection device, characterized in that, include: The acquisition module is used to acquire the image to be detected; The positioning point matching module is used to perform positioning point matching in the image to be detected based on the position information of at least one positioning template region in the preset template image to obtain the positioning point coordinates of at least one target positioning point, wherein each positioning template region corresponds to one target positioning point; the mapping module is used to map the position information of the target detection item region in the preset template image to the image to be detected based on the positioning point coordinates of each target positioning point and the position information of each positioning template region to obtain the detection item mapping region, wherein the target detection item region refers to the region of the item to be detected; the detection module is used to perform correlation detection on the target detection item region in the preset template image and the detection item mapping region in the image to be detected to obtain the defect detection result.

9. A defect detection device, characterized in that, The defect detection device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the defect detection method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the defect detection method as described in any one of claims 1 to 7.

Images