Defect detection method, device, apparatus and storage medium

Abstract

The present disclosure provides a defect detection method, device, equipment and storage medium, by acquiring a component to be tested point cloud and acquiring a target component template point cloud, wherein the target component template point cloud is a reference template point cloud for judging whether the component to be tested point cloud has an attached defect; the component to be tested point cloud and the target component template point cloud are respectively reconstructed into a component to be tested depth map and a target component template depth map; whether the component to be tested point cloud has an attached defect is determined by performing binaryzation processing and Blob analysis on the component to be tested depth map and the target component template depth map, which not only effectively improves the defect detection rate, but also saves a lot of manpower and resources.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to a defect detection method, device, equipment and storage medium. BACKGROUND

[0002] The defect detection method of notebook internal structure based on 3D vision can be understood as a point cloud defect detection problem, specifically, finding a corresponding matching area in a to-be-detected image based on a specified area in a template image through a point cloud registration method, and performing defect detection. The existing technology often uses 3D vision detection methods, including a defect detection method based on deep learning and a defect detection method based on template comparison.

[0003] Among them, the neural network training method based on 3D point cloud has a certain effect on defect detection in theory, but in the actual implementation process, it is very difficult to collect a large number of defect samples, which reduces the detection accuracy.

[0004] And the template comparison defect detection method based on 3D point cloud is to extract the positions of components in the standard template and the to-be-detected point cloud to compare the point cloud information, so as to determine whether the components have an attached defect. This method has the problems of missed detection and false detection, especially when there is a high protrusion near the attached component position or the attached component has a slight offset, the defect detection effect is very poor. SUMMARY

[0005] The present disclosure provides a defect detection method, device, equipment and storage medium to at least solve the above technical problems in the prior art.

[0006] According to a first aspect of the present disclosure, a defect detection method is provided, the method comprising:

[0007] obtaining a component to-be-detected point cloud and obtaining a target component template point cloud, wherein the target component template point cloud is a reference template point cloud for determining whether the component to-be-detected point cloud has an attached defect;

[0008] reconstructing the component to-be-detected point cloud and the target component template point cloud into a component to-be-detected depth map and a target component template depth map, respectively;

[0009] determining whether the component to-be-detected point cloud has an attached defect by performing binarization processing and Blob analysis on the component to-be-detected depth map and the target component template depth map.

[0010] In an implementation manner, the determining whether the component to-be-detected point cloud has an attached defect by performing binarization processing and Blob analysis on the component to-be-detected depth map and the target component template depth map comprises:

[0011] binarize the component to-be-tested depth map and the target component template depth map respectively based on the average depth corresponding to the component template point cloud, to obtain a component to-be-tested binary map and a target component template binary map;

[0012] respectively acquire a first preset region in the component to-be-tested binary map and a second preset region in the target component template binary map;

[0013] compare the region information of the first preset region and the second preset region to determine whether the component to-be-tested point cloud has an attachment defect, wherein the region information includes at least one of the following: width, height, and area.

[0014] In an implementable manner, the comparison of the region information of the first preset region and the second preset region to determine whether the component to-be-tested point cloud has an attachment defect comprises:

[0015] when the width difference of the first preset region and the second preset region is less than a preset width threshold, the height difference of the first preset region and the second preset region is less than a preset height threshold, and the area difference of the first preset region and the second preset region is less than a preset area threshold, it is determined that the component to-be-tested point cloud does not have an attachment defect; otherwise, it is determined that the component to-be-tested point cloud has an attachment defect.

[0016] In an implementable manner, the acquisition of the target component template point cloud comprises:

[0017] acquire a to-be-tested point cloud containing at least one to-be-tested attachment component and acquire a template point cloud of a sample attachment component, wherein the template point cloud is marked with a label of each sample attachment component;

[0018] based on a point cloud registration algorithm and the to-be-tested point cloud containing at least one to-be-tested attachment component, calibrate the template point cloud to obtain a corrected template point cloud;

[0019] on the corrected template point cloud, determine at least one sample component template point cloud, and sequentially take each sample component template point cloud as a target component template point cloud.

[0020] In an implementable manner, the acquisition of the template point cloud of the sample attachment component comprises:

[0021] determine source point cloud data of the sample attachment component by an original 3D design drawing of the sample attachment component or by a 3D camera shooting the sample attachment component;

[0022] model the source point cloud data of the sample attachment component to obtain a template point cloud of the sample attachment component.

[0023] In an implementable manner, the acquisition component is configured to acquire a component to-be-tested point cloud, including:

[0024] The corrected template point cloud and the to-be-tested point cloud containing at least one to-be-tested attached component are subjected to dimension reduction processing to obtain a dimension-reduced template point cloud and a dimension-reduced to-be-tested point cloud.

[0025] In the dimension-reduced template point cloud, a dimension-reduced component template point cloud corresponding to the target component template point cloud is determined.

[0026] Based on a neighborhood minimum method, a dimension-reduced component point cloud corresponding to the dimension-reduced component template point cloud is determined in the dimension-reduced to-be-tested point cloud, and a to-be-tested point cloud region corresponding to the dimension-reduced component point cloud is determined on the to-be-tested point cloud containing at least one to-be-tested attached component as a component to-be-tested point cloud through a mapping relationship between the dimension-reduced to-be-tested point cloud and the to-be-tested point cloud containing at least one to-be-tested attached component.

[0027] In an implementable manner, the respective reconstruction of the component to-be-tested point cloud and the target component template point cloud into a component to-be-tested depth map and a target component template depth map includes:

[0028] The coordinates of each voxel in the component to-be-tested point cloud and the target component template point cloud are respectively acquired, wherein the coordinates include an X value, a Y value and a Z value.

[0029] The X value and the Y value of each voxel in the component to-be-tested point cloud are taken as coordinate values of the component to-be-tested depth map, and the Z value is taken as a pixel value of the component to-be-tested depth map, so as to reconstruct the component to-be-tested depth map.

[0030] The X value and the Y value of each voxel in the target component template point cloud are taken as coordinate values of the target component template depth map, and the Z value is taken as a pixel value of the target component template depth map, so as to reconstruct the target component template depth map.

[0031] According to a second aspect of the present disclosure, a defect detection device is provided, and the device includes:

[0032] A point cloud acquisition module is configured to acquire a component to-be-tested point cloud and acquire a target component template point cloud, wherein the target component template point cloud is a reference template point cloud for judging whether the component to-be-tested point cloud has an attachment defect.

[0033] A depth map reconstruction module is configured to respectively reconstruct the component to-be-tested point cloud and the target component template point cloud into a component to-be-tested depth map and a target component template depth map.

[0034] a defect analysis module configured to determine whether the component to-be-tested point cloud has an attachment defect by binarizing the component to-be-tested depth map and the target component template depth map and performing Blob analysis.

[0035] In an implementation, the defect analysis module is specifically configured to:

[0036] binarize the component to-be-tested depth map and the target component template depth map based on a depth mean value corresponding to the component template point cloud, to obtain a component to-be-tested binary map and a target component template binary map;

[0037] obtain a first preset area in the component to-be-tested binary map and a second preset area in the target component template binary map, respectively;

[0038] compare area information of the first preset area and the second preset area to determine whether the component to-be-tested point cloud has an attachment defect, wherein the area information includes at least one of width, height, and area.

[0039] In an implementation, the defect analysis module is further specifically configured to:

[0040] when a width difference between the first preset area and the second preset area is less than a preset width threshold, a height difference between the first preset area and the second preset area is less than a preset height threshold, and an area difference between the first preset area and the second preset area is less than a preset area threshold, determine that the component to-be-tested point cloud does not have an attachment defect; otherwise, determine that the component to-be-tested point cloud has an attachment defect.

[0041] In an implementation, the point cloud acquisition module is specifically configured to:

[0042] acquire a to-be-tested point cloud containing at least one to-be-tested attachment component and acquire a template point cloud of a sample attachment component, wherein the template point cloud is marked with a label of each sample attachment component;

[0043] calibrate the template point cloud based on a point cloud registration algorithm and the to-be-tested point cloud containing at least one to-be-tested attachment component, to obtain a corrected template point cloud;

[0044] determine at least one sample component template point cloud on the corrected template point cloud, and sequentially take each sample component template point cloud as a target component template point cloud.

[0045] In an implementation, the point cloud acquisition module is further specifically configured to:

[0046] determine source point cloud data of the sample attachment component by an original 3D design drawing of the sample attachment component or by photographing the sample attachment component by a 3D camera.

[0047] Modeling the source point cloud data of the sample attaching component to obtain a template point cloud of the sample attaching component.

[0048] In an implementable manner, the point cloud acquisition module is further specifically configured to:

[0049] Performing dimension reduction processing on the corrected template point cloud and the to-be-tested point cloud containing at least one to-be-tested attaching component to obtain a dimension-reduced template point cloud and a dimension-reduced to-be-tested point cloud.

[0050] Determining, in the dimension-reduced template point cloud, a dimension-reduced component template point cloud corresponding to the target component template point cloud.

[0051] Based on a neighborhood minimum method, determining, in the dimension-reduced to-be-tested point cloud, a dimension-reduced component point cloud corresponding to the dimension-reduced component template point cloud, and determining, on the to-be-tested point cloud containing at least one to-be-tested attaching component, a to-be-tested point cloud region corresponding to the dimension-reduced component point cloud as a component to-be-tested point cloud through a mapping relationship between the dimension-reduced to-be-tested point cloud and the to-be-tested point cloud containing at least one to-be-tested attaching component.

[0052] In an implementable manner, the depth map reconstruction module is specifically configured to:

[0053] Respectively acquiring coordinates of each voxel in the component to-be-tested point cloud and the target component template point cloud, wherein the coordinates include an X value, a Y value and a Z value.

[0054] Taking the X value and the Y value of each voxel in the component to-be-tested point cloud as coordinate values of the component to-be-tested depth map, and taking the Z value as a pixel value of the component to-be-tested depth map, to reconstruct the component to-be-tested depth map.

[0055] Taking the X value and the Y value of each voxel in the target component template point cloud as coordinate values of the target component template depth map, and taking the Z value as a pixel value of the target component template depth map, to reconstruct the target component template depth map.

[0056] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0057] at least one processor; and

[0058] a memory in communication with the at least one processor; wherein

[0059] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the present disclosure.

[0060] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method of the present disclosure.

[0061] The defect detection method, device, equipment and storage medium of the present disclosure, by acquiring the component to be tested point cloud and acquiring the target component template point cloud, wherein the target component template point cloud is a reference template point cloud for judging whether the component to be tested point cloud has an attached defect; the component to be tested point cloud and the target component template point cloud are respectively reconstructed into a component to be tested depth map and a target component template depth map; by performing binaryzation processing and Blob analysis on the component to be tested depth map and the target component template depth map, it is determined whether the component to be tested point cloud has an attached defect, which not only effectively improves the defect detection rate, but also saves a lot of manpower and material resources.

[0062] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0063] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will be more apparent from the following detailed description read in conjunction with the accompanying drawings, in which:

[0064] In the drawings, identical or corresponding reference numerals indicate identical or corresponding parts.

[0065] Figure 1 An implementation flowchart of a defect detection method provided by an embodiment of the present disclosure is shown;

[0066] Figure 2 An implementation flowchart of a defect detection method provided by an embodiment of the present disclosure is shown;

[0067] Figure 3 An implementation flowchart of a defect detection method provided by an embodiment of the present disclosure is shown;

[0068] Figure 4 A structural schematic diagram of a defect detection device provided by an embodiment of the present disclosure is shown;

[0069] Figure 5 A structural schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0070] In order to make the purposes, characteristics and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present disclosure.

[0071] Embodiment one

[0072] Figure 1 A flowchart of a defect detection method provided by the present disclosure is provided, which can be executed by a defect detection device provided by the present disclosure, and the device can be realized in the form of software and / or hardware. The method specifically includes:

[0073] S110, obtaining a component to-be-tested point cloud and obtaining a target component template point cloud.

[0074] The target component template point cloud can be a reference template point cloud for judging whether the to-be-tested component has an attachment defect, denoted as ROI_model. The component to-be-tested point cloud can be a point cloud obtained by using a 3D camera to shoot the to-be-tested component, denoted as ROI_dst.

[0075] Specifically, the attachment positions in the notebook are different, the types of the attached components are different, and the diversity and complexity of the attachment positions themselves seriously affect the detection effect of the attached components. For different types of attached components, for example, the attached components in the notebook can be conductive cloth, foot pads, iron sheets, iron rods, and wires, and due to different manufacturers, the same type of attached component can have different colors. For the diversity and complexity of the attachment positions, for example, the attachment position can be a flat position, an edge corner position, or a position surrounded by interference, and the above factors will affect the detection accuracy.

[0076] In the embodiments of the present disclosure, obtaining the target component template point cloud includes: obtaining a to-be-tested point cloud containing at least one to-be-tested attached component and obtaining a template point cloud of a sample attached component, wherein the template point cloud is marked with labels of each sample attached component; based on a point cloud registration algorithm and the to-be-tested point cloud containing at least one to-be-tested attached component, the template point cloud is calibrated to obtain a corrected template point cloud; at least one sample component template point cloud is determined on the corrected template point cloud, and each sample component template point cloud is sequentially taken as the target component template point cloud.

[0077] The point cloud registration algorithm can be an algorithm for extracting feature points based on voxel segmentation matching. In an example, the point cloud registration algorithm used in this embodiment can be an iterative closest point (ICP) algorithm. The template point cloud can be a source point cloud that has not been calibrated and contains the identities of various sample attachment components, denoted as pt model. The point cloud to be measured is a point cloud containing multiple component point clouds to be measured, which is used to measure whether there is a defect, denoted as pt dst.

[0078] Specifically, in order to obtain an accurate template point cloud, this embodiment needs to first obtain a point cloud to be measured containing at least one attachment component to be measured, and a template point cloud with the identities of various sample attachment components. In theory, if the template point cloud and the point cloud to be measured are captured under the same detection mechanism, the same position, the same camera, and the same design parameters, etc., there is no need for correction. However, in actual operation, due to the possible small deviations in the above-mentioned links, as well as the interference of factors such as design accuracy deviation, the template point cloud and the point cloud to be measured may deviate. Therefore, in order to solve the above-mentioned problems, this embodiment calibrates the template point cloud based on a point cloud registration algorithm and a point cloud containing at least one attachment component to be measured, so that the template point cloud is corrected according to the point cloud to be measured, thereby obtaining an accurately calibrated template point cloud, denoted as pt model'.

[0079] Specifically, since there is at least one sample attachment component point cloud on the calibrated template point cloud in this embodiment, this embodiment can sequentially extract and segment each sample attachment component point cloud, and take the segmented independent component as a target component template point cloud in turn, which facilitates subsequent comparison with the corresponding attachment component point cloud to be measured.

[0080] In the embodiments of the present disclosure, the template point cloud of the sample attachment component is obtained, including: determining the source point cloud data of the sample attachment component by an original 3D design drawing of the sample attachment component or by a 3D camera capturing the sample attachment component; and modeling the source point cloud data of the sample attachment component to obtain the template point cloud of the sample attachment component.

[0081] The source point cloud data can be obtained by a 3D camera or a 3D design drawing, and contains point cloud data of all sample attachment components.

[0082] Specifically, to obtain the template point cloud of the sample attachment components, this embodiment uses the original 3D design drawing of the sample attachment components or images of the sample attachment components captured by a 3D camera to obtain source point cloud data containing all sample attachment components. This source point cloud data is then processed through modeling to obtain the template point cloud containing each attachment component, i.e., the template point cloud of the sample attachment components. Each sample attachment component is then marked to facilitate subsequent segmentation and comparison. For example, in this embodiment, if the source point cloud data of the sample attachment components is obtained by direct capture with a 3D camera, modeling can be performed manually, and each attachment component to be tested can be marked.

[0083] In this embodiment of the disclosure, obtaining the component test point cloud includes: performing dimensionality reduction processing on the corrected template point cloud and the test point cloud containing at least one attached component to be tested, to obtain a dimensionality-reduced template point cloud and a dimensionality-reduced test point cloud; within the dimensionality-reduced template point cloud, determining the dimensionality-reduced component template point cloud corresponding to the target component template point cloud; based on the neighborhood minimum method, determining the dimensionality-reduced component point cloud corresponding to the dimensionality-reduced component template point cloud within the dimensionality-reduced test point cloud, and through the mapping relationship between the dimensionality-reduced test point cloud and the test point cloud containing at least one attached component to be tested, determining the test point cloud region corresponding to the dimensionality-reduced component point cloud as the component test point cloud on the test point cloud containing at least one attached component to be tested.

[0084] The dimensionality-reduced template point cloud can be a template point cloud with a Z-axis coordinate value of 0, denoted as pt_model”. The dimensionality-reduced test point cloud can be a test point cloud with a Z-axis coordinate value of 0, denoted as pt_dst”. The neighborhood minimum method can be a method used to match the dimensionality-reduced test point cloud with the corresponding dimensionality-reduced component template point cloud. For example, the neighborhood minimum method used in this embodiment is the 8-neighborhood minimum method. The dimensionality-reduced component template point cloud refers to the independent component region segmented within the dimensionality-reduced template point cloud, denoted as ROI_model”[N]. The dimensionality-reduced component point cloud can be an attached component point cloud with a Z-axis coordinate value of 0, corresponding to the dimensionality-reduced component template point cloud, denoted as ROI_dst”[N]. The component test point cloud can be a point cloud region on the test point cloud that corresponds to the dimensionality-reduced component point cloud, denoted as ROI_dst[N].

[0085] Specifically, due to the large amount of information of the point cloud, the system operation speed is slow, and due to the interference of the height difference of the notebook internal structure itself, the system judgment error is easy to cause, therefore, in order to improve the operation efficiency of the system and reduce the judgment error rate of the system, the embodiment carries out dimension reduction processing on the obtained corrected template point cloud and the to-be-tested point cloud containing at least one to-be-tested attached component, that is, the Z-axis coordinate value of these point clouds is set to 0, so as to obtain the template point cloud with the Z-axis coordinate value of 0 and the to-be-tested point cloud containing at least one to-be-tested attached component with the Z-axis coordinate value of 0, and obtain the dimension reduction template point cloud and the dimension reduction to-be-tested point cloud. Since the dimension reduction template point cloud is an integral template, in the dimension reduction template point cloud, the corresponding dimension reduction component template point cloud can be determined according to the selected local target component template point cloud. In the dimension reduction to-be-tested point cloud, the embodiment uses the neighborhood minimum value method to obtain the dimension reduction component point cloud corresponding to the dimension reduction component template point cloud according to the X coordinate and Y coordinate of the dimension reduction component template point cloud and the dimension reduction component point cloud. Then, the embodiment determines the to-be-tested point cloud region corresponding to the dimension reduction component point cloud on the to-be-tested point cloud, that is, the component to-be-tested point cloud, according to the mapping relationship between the dimension reduction to-be-tested point cloud and the to-be-tested point cloud containing at least one to-be-tested attached component.

[0086] The embodiment can eliminate the interference of the height of the notebook internal structure itself, and reduce the possibility of judgment error.

[0087] S120, respectively reconstruct the component to-be-tested point cloud and the target component template point cloud into a component to-be-tested depth map and a target component template depth map.

[0088] Wherein, the component to-be-tested depth map can be obtained by the component to-be-tested point cloud, and is used to generate a component to-be-tested binary image, denoted as dep_dst. The target component template depth map can be obtained by the target component template point cloud, and is used to generate a target component template binary image, denoted as dep_model.

[0089] In the embodiments of the present disclosure, reconstructing the component to-be-tested point cloud and the target component template point cloud into a component to-be-tested depth map and a target component template depth map respectively comprises: acquiring coordinates of each voxel in the component to-be-tested point cloud and the target component template point cloud respectively, wherein the coordinates comprise an X value, a Y value and a Z value; taking the X value and the Y value of each voxel in the component to-be-tested point cloud as coordinate values of the component to-be-tested depth map, and taking the Z value as a pixel value of the component to-be-tested depth map, so as to reconstruct the component to-be-tested depth map; and taking the X value and the Y value of each voxel in the target component template point cloud as coordinate values of the target component template depth map, and taking the Z value as a pixel value of the target component template depth map, so as to reconstruct the target component template depth map.

[0090] Specifically, in order to determine whether the to-be-tested component has an abnormal attachment, the embodiments need to convert the component to-be-tested point cloud and the target component template point cloud into a component to-be-tested depth map and a target component template depth map that can be subjected to binarization processing and Blob analysis operation. Specifically, first, the X, Y and Z axis coordinate values of each voxel corresponding to the component to-be-tested point cloud and the target component template point cloud are obtained, and then the X and Y axis coordinate values of each voxel in the component to-be-tested point cloud are taken as the coordinate values of the component to-be-tested depth map after dimension reduction, and the Z axis coordinate value of each voxel is taken as the pixel value of the component to-be-tested depth map, so as to obtain the reconstructed component to-be-tested depth map. Similarly, the X and Y axis coordinate values of each voxel in the target component template point cloud are taken as the coordinate values of the target component template depth map after dimension reduction, and the Z axis coordinate value of each voxel is taken as the pixel value of the target component template depth map, so as to obtain the reconstructed target component template depth map.

[0091] In S130, whether the component to-be-tested point cloud has an attachment defect is determined by binarizing the component to-be-tested depth map and the target component template depth map and performing Blob analysis.

[0092] The binarization processing can be an operation of converting the component to-be-tested depth map and the target component template depth map into a grayscale map. The Blob analysis can be an operation of analyzing physical features such as width, height and area of the component to-be-tested binary map and the target component template binary map. In the embodiments, the Blob analysis method can be FindContours in the Open cv algorithm library.

[0093] Specifically, in order to determine whether the component to-be-tested point cloud has an attachment defect, the component to-be-tested depth map and the target component template depth map obtained are subjected to binarization processing, so as to obtain a component to-be-tested binary map and a target component template binary map, and then the component to-be-tested binary map and the target component template binary map obtained are subjected to Blob analysis, so as to determine whether the component to-be-tested point cloud has an attachment defect.

[0094] The existing defect detection method is a neural network training method based on 3D point cloud. The method has certain effect on detecting whether there is an attached defect, but a large amount of manpower is needed to collect, label and train samples. This method is not only time-consuming and labor-intensive, but also inconvenient to operate. In addition, the method has good defect detection effect on parts with standard attachment and no height protrusion around the attachment position, but has poor defect detection effect on parts with height protrusion near the attachment position or with slight offset of the attached part. The method adopted in the embodiment not only effectively solves the problem of low detection accuracy when there is height protrusion near the attachment position of the attached part or the attached part has slight offset, but also makes up for the shortcomings of the neural network training method based on 3D point cloud, such as wasting a large amount of manpower and material resources. In addition, the method effectively improves the detection speed of the algorithm.

[0095] Embodiment Two

[0096] Figure 2 A flowchart of a defect detection method provided by Embodiment Two of the present disclosure is provided. Based on the above-mentioned embodiments, the present disclosure determines whether the part to be tested point cloud has an attached defect by performing binaryzation processing and Blob analysis on the part to be tested depth map and the target part template depth map, which includes: based on the depth mean value corresponding to the part template point cloud, respectively performing binaryzation processing on the part to be tested depth map and the target part template depth map to obtain the part to be tested binary map and the target part template binary map; respectively acquiring a first preset area in the part to be tested binary map and a second preset area in the target part template binary map; comparing the area information of the first preset area and the second preset area to determine whether the part to be tested point cloud has an attached defect, wherein the area information includes at least one of the following: width, height and area. The method specifically includes:

[0097] S210, acquiring the part to be tested point cloud and acquiring the target part template point cloud.

[0098] S220, respectively reconstructing the part to be tested point cloud and the target part template point cloud into the part to be tested depth map and the target part template depth map.

[0099] S230, based on the depth mean value corresponding to the part template point cloud, respectively performing binaryzation processing on the part to be tested depth map and the target part template depth map to obtain the part to be tested binary map and the target part template binary map.

[0100] Wherein, the part to be tested binary map can be a gray scale map obtained by binaryzation processing of the part to be tested point cloud, denoted as bin_dep_model. The target part template binary map can be a gray scale map obtained by binaryzation processing of the target part template point cloud, denoted as bin_dep_dst.

[0101] Specifically, in order to simplify the operation and improve the work efficiency, the Z-axis coordinate value of the component template point cloud is obtained, so that the average depth value corresponding to the component template point cloud can be obtained, and then the obtained component to-be-tested depth map and the target component template depth map are processed by binarization based on the average depth value of the component template point cloud, so that the corresponding component to-be-tested binary map and the target component template binary map are obtained. The component to-be-tested binary map and the target component template binary map are black and white grayscale maps.

[0102] In S240, a first preset region in the component to-be-tested binary map and a second preset region in the target component template binary map are obtained respectively.

[0103] The first preset region can be a white region in the component to-be-tested binary map. The second preset region can be a white region in the target component template binary map.

[0104] In this embodiment, the average depth value corresponding to the component template point cloud is taken as a reference to convert the component to-be-tested binary map and the target component template binary map. Since the attached component region is higher than the region of the internal structure of the notebook computer, the white region is displayed in the above-mentioned binary map. In this embodiment, the white region in the component to-be-tested binary map can be directly taken as the first preset region, and the white region in the target component template binary map can be directly taken as the second preset region.

[0105] In S250, the region information of the first preset region and the second preset region is compared to determine whether the component to-be-tested point cloud has an attachment defect.

[0106] The region information includes at least one of the width, the height, and the area.

[0107] Specifically, the first preset region in the obtained component to-be-tested binary map and the second preset region in the target component template binary map are compared by means of Blob analysis, that is, the width, the height, and the area of the white region in the first preset region and the second preset region are compared, so as to determine whether the attached component in the component to-be-tested binary map has a defect, and further determine whether the component to-be-tested point cloud has an attachment defect.

[0108] In the embodiment of the present disclosure, comparing the region information of the first preset region and the second preset region to determine whether the component to-be-tested point cloud has an attachment defect includes: when the width difference value of the first preset region and the second preset region is less than a preset width threshold value, the height difference value of the first preset region and the second preset region is less than a preset height threshold value, and the area difference value of the first preset region and the second preset region is less than a preset area threshold value, it is determined that the component to-be-tested point cloud does not have an attachment defect; otherwise, it is determined that the component to-be-tested point cloud has an attachment defect.

[0109] The preset width threshold, the preset height threshold and the preset area threshold can be any value set according to actual conditions, and the embodiment does not limit them.

[0110] Specifically, the width, height and area of the white region in the obtained first preset region are compared with the width, height and area of the white region in the second preset region. If the difference between the width, height and area of the white region in the first preset region and the corresponding second preset region is less than the corresponding preset threshold, it is determined that the component to be tested point cloud does not have an attachment defect. If any one of the difference between the width, height and area of the white region in the first preset region and the corresponding second preset region is greater than or equal to the corresponding preset threshold, it is determined that the component to be tested point cloud has an attachment defect.

[0111] Figure 3 The implementation flow framework diagram of the defect detection method provided by the second embodiment of the present disclosure. Since the existing defect detection method has good defect detection effect on standard attachment and no height protrusion around the attachment position of the component, but has poor defect detection effect when there is a height protrusion near the attachment position or a slight offset of the attachment, therefore, the embodiment provides an effective solution to the above problems, and the detailed steps are as follows:

[0112] 1. Read the 3D template point cloud pt_model as the source point cloud, read the 3D to-be-tested point cloud pt_dst as the target point cloud, perform point cloud registration, and obtain the 3D conversion matrix T;

[0113] 2. Cross-multiply the 3D template point cloud pt_model and the 3D conversion matrix T to obtain the corrected template point cloud pt_model', as shown in formula [1];

[0114] pt_model' = pt_model x T[1]

[0115] 3. Extract the attachment component region of the template point cloud pt_model', and segment it into independent component regions ROI_model[N] (i.e. target component template point cloud), wherein N is the total number of independent component regions contained in the template information;

[0116] 4. Respectively, the template point cloud pt_model' and the to-be-tested point cloud pt_dst are processed by dimension reduction, i.e. the z coordinate value is set to 0, to obtain the dimension-reduced template point cloud pt_model" and the dimension-reduced to-be-tested point cloud pt_dst";

[0117] 5. Extract the attached component region of the dimension-reduced template point cloud pt_model, and segment it into individual independent component regions ROI_model[N] (i.e., dimension-reduced component template point cloud), N being the total number of independent component regions contained in the template information;

[0118] 6. Based on each independent component ROI_model[N], obtain the corresponding component point cloud ROI_dst[N] (i.e., dimension-reduced component point cloud) in the dimension-reduced point cloud pt_dst by the 8-neighbor minimum method, and further map it to the point cloud pt_dst, so as to obtain the component point cloud ROI_dst[N] corresponding to the target component template point cloud ROI_model[N];

[0119] 7. Convert the target component template point cloud ROI_model[i] and the component point cloud ROI_dst[i] into a depth map dep_model[i] (i.e., target component template depth map) and dep_dst[i] (i.e., component test depth map) respectively, wherein i is any independent component region.

[0120] 8. Based on the depth in the target component template point cloud ROI_model[i], binarize the target component template depth map dep_model[i] and the component test depth map dep_dst[i] to obtain the component test binary map bin_dep_model[i] and the target component template binary map bin_dep_dst[i]. The calculation method is as shown in formula [2] and [3]:

[0121]

[0122] wherein bin_dep_model[i](x, y) is the pixel value of the template depth binary map, bin_dep_dst[i](x, y) is the pixel value of the test depth binary map, ROI_model[i]_z is the z-axis depth value of the target component template point cloud, dep_dst[i](x, y) is the depth value of each pixel in the component test depth map, and mean represents the average value;

[0123] 9. Perform Blob analysis on the bin_dep_model[i] binary map and the bin_dep_dst[i] binary map respectively to obtain the width, height, area, and other information of the white region;

[0124] 10. When the width, height, and area of the white region of the template and test binary maps are all relatively close, it is considered that the test region is normally attached with the component; otherwise, it is considered that the test region is not attached with the component. The calculation method is as shown in formula [4];

[0125]

[0126] wherein flag_pass is a normal flag bit, 0 represents abnormality, 1 represents normality, w_mdl, h_mdl, area_mdl are the width, height and area of the white region of the template depth binary image respectively, w_dst, h_dst, area_dst are the width, height and area of the white region of the to-be-tested depth binary image respectively, T_w, T_h, T_area are the width threshold, height threshold and area threshold of the white region of the binary image respectively.

[0127] The embodiment not only effectively solves the problem of low detection accuracy when there is a high protrusion near the position of the attached component or the attached component has a slight deviation, but also makes up for the shortcomings of the 3D point cloud-based neural network training method, such as wasting a large amount of manpower and material resources, and improves the detection efficiency.

[0128] Embodiment Three

[0129] Figure 4 is a structural schematic diagram of a defect detection device provided by the embodiment of the disclosure, and the device specifically comprises:

[0130] The point cloud acquisition module 410 is configured to acquire the component to-be-tested point cloud and acquire a target component template point cloud, wherein the target component template point cloud is a reference template point cloud for judging whether the component to-be-tested point cloud has an attachment defect.

[0131] The depth image reconstruction module 420 is configured to reconstruct the component to-be-tested point cloud and the target component template point cloud into a component to-be-tested depth image and a target component template depth image respectively.

[0132] The defect analysis module 430 is configured to determine whether the component to-be-tested point cloud has an attachment defect by performing binaryzation processing and Blob analysis on the component to-be-tested depth image and the target component template depth image.

[0133] In an implementable manner, the defect analysis module 430 is specifically configured to:

[0134] perform binaryzation processing on the component to-be-tested depth image and the target component template depth image respectively based on the depth mean value corresponding to the component template point cloud, to obtain a component to-be-tested binary image and a target component template binary image;

[0135] acquire a first preset region in the component to-be-tested binary image and a second preset region in the target component template binary image respectively;

[0136] compare the region information of the first preset region and the second preset region to determine whether the component to-be-tested point cloud has an attachment defect, wherein the region information comprises at least one of the following: width, height and area.

[0137] In an implementable manner, the defect analysis module 430 is further specifically configured to:

[0138] When the width difference between the first preset region and the second preset region is less than a preset width threshold, the height difference between the first preset region and the second preset region is less than a preset height threshold, and the area difference between the first preset region and the second preset region is less than a preset area threshold, it is determined that the component to-be-tested point cloud does not have the attachment defect; otherwise, it is determined that the component to-be-tested point cloud has the attachment defect.

[0139] In an implementable manner, the point cloud acquisition module 410 is specifically configured to:

[0140] acquire the to-be-tested point cloud containing at least one to-be-tested attachment component and acquire a template point cloud of a sample attachment component, wherein the template point cloud is marked with labels of the respective sample attachment components;

[0141] based on a point cloud registration algorithm and the to-be-tested point cloud containing at least one to-be-tested attachment component, calibrate the template point cloud to obtain a corrected template point cloud;

[0142] on the corrected template point cloud, determine at least one sample component template point cloud, and sequentially take each sample component template point cloud as a target component template point cloud.

[0143] In an implementable manner, the point cloud acquisition module 410 is further specifically configured to:

[0144] determine source point cloud data of the sample attachment component by an original 3D design drawing of the sample attachment component or by shooting the sample attachment component through a 3D camera;

[0145] model the source point cloud data of the sample attachment component to obtain a template point cloud of the sample attachment component.

[0146] In an implementable manner, the point cloud acquisition module 410 is further specifically configured to:

[0147] perform dimension reduction processing on the corrected template point cloud and the to-be-tested point cloud containing at least one to-be-tested attachment component to obtain a dimension-reduced template point cloud and a dimension-reduced to-be-tested point cloud;

[0148] in the dimension-reduced template point cloud, determine a dimension-reduced component template point cloud corresponding to the target component template point cloud;

[0149] based on a neighborhood minimum value method, in the dimension-reduced to-be-tested point cloud, determine a dimension-reduced component point cloud corresponding to the dimension-reduced component template point cloud, and through a mapping relationship between the dimension-reduced to-be-tested point cloud and the to-be-tested point cloud containing at least one to-be-tested attachment component, determine a to-be-tested point cloud region corresponding to the dimension-reduced component point cloud on the to-be-tested point cloud containing at least one to-be-tested attachment component as a component to-be-tested point cloud.

[0150] In an implementable manner, the depth map reconstruction module 420 is specifically configured to:

[0151] respectively acquire coordinates of each voxel in the component to-be-tested point cloud and the target component template point cloud, wherein the coordinates include X value, Y value and Z value;

[0152] take the X value and the Y value of each voxel in the component to-be-tested point cloud as coordinate values of the component to-be-tested depth map, and take the Z value as a pixel value of the component to-be-tested depth map, to reconstruct the component to-be-tested depth map;

[0153] take the X value and the Y value of each voxel in the target component template point cloud as coordinate values of the target component template depth map, and take the Z value as a pixel value of the target component template depth map, to reconstruct the target component template depth map.

[0154] According to embodiments of the present disclosure, the present disclosure further provides an electronic device and a readable storage medium.

[0155] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0156] As shown in Figure 5 The electronic device 500 includes a computing unit 501 that can perform various appropriate actions and processes in accordance with computer programs stored in a read-only memory (ROM) 502 or loaded into a random access memory (RAM) 503 from a storage unit 508. Various programs and data required for operation of the electronic device 500 can also be stored in the RAM 503. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0157] A plurality of components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a magnetic disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0158] The computing unit 501 can be various general and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 performs various methods and processes described above, such as the defect detection method. For example, in some embodiments, the defect detection method can be implemented as a computer software program, which is tangibly embodied in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded onto the RAM 503 and executed by the computing unit 501, one or more steps of the defect detection method described above can be performed. Alternatively, in other embodiments, the computing unit 501 can be configured to perform the defect detection method by any other appropriate means, such as by means of firmware.

[0159] Various implementations of the systems and techniques described above herein can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0160] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a standalone software package, or entirely on a remote machine or server.

[0161] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0162] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0163] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0164] The computer system can include clients and servers. The clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server can arise by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The servers can be cloud servers, servers of a distributed system, or servers combined with a blockchain.

[0165] It should be understood that the steps shown in the various forms above can be reordered, added to, or deleted from. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, as long as the desired results of the technical solutions of the present disclosure are achieved, and the present disclosure is not limited herein.

[0166] In addition, the terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0167] The above description is merely a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present disclosure, which should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A defect detection method characterized by, The method comprises: obtaining a component to be tested point cloud and obtaining a target component template point cloud, wherein the target component template point cloud is a reference template point cloud for judging whether the component to be tested point cloud has an attachment defect; reconstructing the component to be tested point cloud and the target component template point cloud into a component to be tested depth map and a target component template depth map respectively; determining whether the component to be tested point cloud has an attachment defect through binarization processing and Blob analysis on the component to be tested depth map and the target component template depth map; wherein the obtaining of the target component template point cloud comprises: obtaining a to-be-tested point cloud containing at least one to-be-tested attachment component and obtaining a template point cloud of a sample attachment component, wherein the template point cloud is marked with labels of each sample attachment component; calibrating the template point cloud based on a point cloud registration algorithm and the to-be-tested point cloud containing at least one to-be-tested attachment component to obtain a corrected template point cloud; determining at least one sample component template point cloud on the corrected template point cloud, and sequentially taking each sample component template point cloud as a target component template point cloud; wherein the obtaining of the template point cloud of the sample attachment component comprises: determining source point cloud data of the sample attachment component through an original 3D design drawing of the sample attachment component or through a 3D camera shooting the sample attachment component; modeling the source point cloud data of the sample attachment component to obtain the template point cloud of the sample attachment component; wherein the obtaining of the component to be tested point cloud comprises: performing dimension reduction processing on the corrected template point cloud and the to-be-tested point cloud containing at least one to-be-tested attachment component to obtain a reduced dimension template point cloud and a reduced dimension to-be-tested point cloud; determining a reduced dimension component template point cloud corresponding to the target component template point cloud in the reduced dimension template point cloud; determining a reduced dimension component point cloud corresponding to the reduced dimension component template point cloud in the reduced dimension to-be-tested point cloud based on a neighborhood minimum method, and determining a to-be-tested point cloud region corresponding to the reduced dimension component point cloud on the to-be-tested point cloud containing at least one to-be-tested attachment component as a component to be tested point cloud through a mapping relationship between the reduced dimension to-be-tested point cloud and the to-be-tested point cloud containing at least one to-be-tested attachment component.

2. The method of claim 1, wherein, The determining of whether the component to be tested point cloud has an attachment defect through binarization processing and Blob analysis on the component to be tested depth map and the target component template depth map comprises: performing binarization processing on the component to be tested depth map and the target component template depth map respectively based on a depth mean value corresponding to a component template point cloud to obtain a component to be tested binary image and a target component template binary image; obtaining a first preset region in the component to be tested binary image and a second preset region in the target component template binary image respectively; comparing region information of the first preset region and the second preset region to determine whether the component to be tested point cloud has an attachment defect, wherein the region information contains at least one of width, height, and area.

3. The method of claim 2, wherein, The comparing the region information of the first preset region and the second preset region to determine whether the component to-be-tested point cloud has an attachment defect comprises: When a width difference between the first preset region and the second preset region is less than a preset width threshold, a height difference between the first preset region and the second preset region is less than a preset height threshold, and an area difference between the first preset region and the second preset region is less than a preset area threshold, it is determined that the component to-be-tested point cloud does not have an attachment defect; otherwise, it is determined that the component to-be-tested point cloud has an attachment defect.

4. The method of claim 1, wherein, The reconstructing the component to-be-tested point cloud and the target component template point cloud into a component to-be-tested depth map and a target component template depth map, respectively, comprises: coordinates of each voxel in the component to-be-tested point cloud and the target component template point cloud are acquired, respectively, wherein the coordinates comprise an X value, a Y value, and a Z value; the X value and the Y value of each voxel in the component to-be-tested point cloud are taken as coordinate values of the component to-be-tested depth map, and the Z value is taken as a pixel value of the component to-be-tested depth map, so as to reconstruct the component to-be-tested depth map; the X value and the Y value of each voxel in the target component template point cloud are taken as coordinate values of the target component template depth map, and the Z value is taken as a pixel value of the target component template depth map, so as to reconstruct the target component template depth map.

5. A defect detection apparatus characterized by comprising: The device comprises: a point cloud acquisition module configured to acquire a component to-be-tested point cloud and a target component template point cloud, wherein the target component template point cloud is a reference template point cloud for determining whether the component to-be-tested point cloud has an attachment defect; a depth map reconstruction module configured to reconstruct the component to-be-tested point cloud and the target component template point cloud into a component to-be-tested depth map and a target component template depth map, respectively; a defect analysis module configured to determine whether the component to-be-tested point cloud has an attachment defect by performing binaryzation processing and Blob analysis on the component to-be-tested depth map and the target component template depth map. The point cloud acquisition module is also configured to acquire a test point cloud containing at least one test attaching component and a template point cloud of a sample attaching component, wherein the template point cloud is marked with labels of respective sample attaching components; based on a point cloud registration algorithm and the test point cloud containing at least one test attaching component, the template point cloud is calibrated to obtain a corrected template point cloud; at least one sample component template point cloud is determined on the corrected template point cloud, and each sample component template point cloud is sequentially taken as a target component template point cloud; source point cloud data of the sample attaching component is determined by an original 3D design drawing of the sample attaching component or by a 3D camera shooting the sample attaching component; the source point cloud data of the sample attaching component is modeled to obtain the template point cloud of the sample attaching component; the corrected template point cloud and the test point cloud containing at least one test attaching component are subjected to dimension reduction processing to obtain a dimension-reduced template point cloud and a dimension-reduced test point cloud; a dimension-reduced component template point cloud corresponding to the target component template point cloud is determined in the dimension-reduced template point cloud; based on a neighborhood minimum method, a dimension-reduced component point cloud corresponding to the dimension-reduced component template point cloud is determined in the dimension-reduced test point cloud, and a test point cloud region corresponding to the dimension-reduced component point cloud is determined as a component test point cloud on the test point cloud containing at least one test attaching component through a mapping relationship between the dimension-reduced test point cloud and the test point cloud containing at least one test attaching component.

6. An electronic device, comprising: Comprise: At least one processor; And The memory is in communication connection with the at least one processor; wherein The memory stores instructions executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of any one of claims 1-4.

7. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to execute the method of any one of claims 1-4.

Images