Defect detection method, device, electronic equipment and storage medium

Abstract

The present invention discloses a defect detection method, device, electronic device and storage medium. The method comprises: obtaining original three-dimensional point cloud data of a plane to be detected; determining a target reference plane of the plane to be detected based on the original three-dimensional point cloud data; wherein the number of internal points in the original three-dimensional point cloud data whose distance to the target reference plane is less than a preset distance threshold is the largest; performing point cloud reconstruction on the original three-dimensional point cloud data based on the target reference plane to generate target three-dimensional point cloud data; and performing defect detection on the plane to be detected based on the target three-dimensional point cloud data. Through the technical solution implemented by the present invention, the defect area of ​​the plane to be detected can be accurately detected to meet the high-precision detection scenario.

Description

Technical Field

[0001] The present invention relates to the field of defect detection, and in particular to a defect detection method, device, electronic equipment and storage medium. Background Art

[0002] With the popularization and rapid development of computer vision technology, computer vision is increasingly used in scenarios such as product defect detection in industrial production, especially in the field of new energy power batteries with demanding process requirements. Such scenarios include 3D defect detection of sealing pin welding of new energy vehicle batteries, busbar welding defect detection, and bottom bubble detection of battery cell modules. For the above detection scenarios, it is necessary to accurately detect defect areas such as pits or protrusions on the plane.

[0003] In the prior art, defective areas are mainly measured through 2.5D photometric stereo imaging. The principle of this method is that when a 2D camera is illuminated by light sources at different angles, the height changes of the defect in the image taken will be reflected as changes in the brightness of the image pixels. Multiple images at different angles are finally synthesized into an image that reflects the changes in brightness (large changes in brightness indicate the presence of defects), and defect detection is performed based on this image.

[0004] The disadvantages of this solution are that the detection background is complex, it is easily affected by ambient light, and the detection accuracy is low. Summary of the invention

[0005] The present invention provides a defect detection method, device, electronic device and storage medium. Through the technical solution of the present invention, the defect area of ​​the plane to be detected can be accurately detected to meet the high-precision detection scenario.

[0006] According to one aspect of the present invention, a defect detection method is provided, the method comprising:

[0007] Obtaining original three-dimensional point cloud data of the plane to be detected;

[0008] Determining a target reference plane of the plane to be detected according to the original three-dimensional point cloud data; wherein the number of inliers in the original three-dimensional point cloud data whose distance to the target reference plane is less than a preset distance threshold is the largest;

[0009] Reconstructing the original three-dimensional point cloud data based on the target reference plane to generate target three-dimensional point cloud data;

[0010] Defect detection is performed on the plane to be detected based on the target three-dimensional point cloud data.

[0011] According to another aspect of the present invention, there is provided a defect detection device, the device comprising:

[0012] A three-dimensional point cloud data acquisition module is used to acquire original three-dimensional point cloud data of the plane to be detected;

[0013] A target reference plane determination module is used to determine the target reference plane of the plane to be detected according to the original three-dimensional point cloud data; wherein the number of the inner points in the original three-dimensional point cloud data whose distance to the target reference plane is less than a preset distance threshold is the largest;

[0014] A three-dimensional point cloud data reconstruction module is used to reconstruct the original three-dimensional point cloud data based on the target reference plane to generate target three-dimensional point cloud data;

[0015] A defect detection module is used to perform defect detection on the plane to be detected based on the target three-dimensional point cloud data.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the defect detection method described in any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the defect detection method described in any embodiment of the present invention when executed.

[0021] In the embodiment of the present invention, the original three-dimensional point cloud data of the plane to be detected is obtained, and the microscopic defects that cannot be captured by the human eye can be converted into a data set that can be calculated; the target reference plane of the plane to be detected is determined according to the original three-dimensional point cloud data; wherein, the number of internal points in the original three-dimensional point cloud data whose distance to the target reference plane is less than a preset distance threshold is the largest, and the target reference plane with the highest degree of fit with the plane to be detected can be determined; the original three-dimensional point cloud data is reconstructed based on the target reference plane to generate target three-dimensional point cloud data, and the plane where the reconstructed target three-dimensional point cloud data is located can coincide with the XY plane in the three-dimensional coordinate system of the space, so as to facilitate subsequent processing; the plane to be detected is detected for defects based on the target three-dimensional point cloud data. The technical solution implemented by the present invention can not only solve the technical problem of complex detection background and susceptibility to ambient light when performing defect area detection in 2.5D photometric stereo imaging, but also accurately detect the defect area of ​​the plane to be detected to meet the high-precision detection scenario.

[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 is a flow chart of a defect detection method provided according to Embodiment 1 of the present invention;

[0025] Figure 2 is a schematic diagram of a laser triangulation measurement principle provided according to the first embodiment of the present invention;

[0026] Figure 3a A physical battery provided according to the first embodiment of the present invention;

[0027] Figure 3b A three-dimensional point cloud data map of a battery bottom provided according to the first embodiment of the present invention;

[0028] Figure 3c is a side view of three-dimensional point cloud data of a battery bottom provided according to the first embodiment of the present invention;

[0029] Figure 4 is a flow chart of a defect detection method provided according to Embodiment 2 of the present invention;

[0030] Figure 5a A three-dimensional point cloud data map of the bottom of a battery after reconstruction provided by Embodiment 2 of the present invention;

[0031] Figure 5b is a side view of the reconstructed three-dimensional point cloud data of the bottom of a battery provided by the second embodiment of the present invention;

[0032] Figure 6 3D point cloud data segmentation schematic diagram based on Z value provided by Embodiment 2 of the present invention;

[0033] Figure 7a A target three-dimensional point cloud data map of a segmented concave defect area provided by the second embodiment of the present invention;

[0034] Figure 7b is a side view of target three-dimensional point cloud data of a segmented concave defect area provided by Embodiment 2 of the present invention;

[0035] Figure 8 is a flow chart of measuring a defective area provided according to the second embodiment of the present invention;

[0036] Figure 9a is a top view of target three-dimensional point cloud data of a segmented concave defect area provided by the second embodiment of the present invention;

[0037] Figure 9b is an 8-bit grayscale image of a concave defect area provided according to the second embodiment of the present invention;

[0038] Fig.10 is a structural schematic diagram of a defect detection device provided according to Embodiment 3 of the present invention;

[0039] Fig.11 It is a schematic diagram of the structure of an electronic device for implementing the defect detection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0041] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0042] Embodiment 1

[0043] Figure 1 A flowchart of a defect detection method is provided for the first embodiment of the present invention. This embodiment is applicable to various types of planar 3D defect detection situations. The method can be performed by a defect detection device. The defect detection device can be implemented in the form of hardware and / or software. The defect detection device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0044] S110, obtaining original three-dimensional point cloud data of the plane to be detected.

[0045] Among them, raw 3D point cloud data refers to a set of unprocessed 3D point data obtained from scanning, cameras or other sensors. These points are usually composed of xyz coordinates and other attributes such as color, normal, etc. The raw 3D point cloud data can be used for 3D reconstruction, object recognition, robot navigation, etc. The plane to be inspected refers to the plane where defects will be detected.

[0046] In an embodiment of the present invention, before performing defect detection on a plane to be detected, it is necessary to first obtain the original three-dimensional point cloud data of the plane to be detected. Thus, microscopic defects that cannot be captured by the human eye can be converted into a data set that can be calculated. Specifically, obtaining the original three-dimensional point cloud data of the plane to be detected includes:

[0047] The original three-dimensional point cloud data of the plane to be detected is obtained based on the principle of laser triangulation.

[0048] Among them, the laser triangulation principle is a measurement method based on geometry, which uses triangular geometric relationships to calculate the distance of objects. Figure 2 A schematic diagram of laser triangulation is shown, Figure 2As shown, the angle between the incident light AO and the baseline AB is α, AB is the distance between the center of the laser and the center of the photosensitive unit, BF is the focal length f of the lens, and D is the limit position of the reflected light imaged on the photosensitive unit when the measured object is infinitely far away from the baseline. DE is the displacement of the light spot on the photosensitive unit from the limit position, recorded as x. When the optical path of the system is determined, α, AB and f are all known parameters. From the geometric relationship in the optical path diagram, it can be seen that △ABO∽△DEB, then there is a side length relationship:

[0049] and Then you can get

[0050] When determining the optical path of the system, one axis of the photosensitive unit position sensor can be parallel to the baseline AB (assuming it is the y-axis), and the pixel coordinates of the laser light spot obtained by the algorithm are (P X , P y ) can get the value of x as:

[0051] x=PixelSize·P X +DeviationValue

[0052] Among them, PixeSize is the size of a single pixel on the photosensitive unit, and DeviationValue is the deviation between the projection distance calculated by the pixel point and the actual projection distance x.

[0053] When the measured object has a relative displacement with the baseline AB, x changes to x'. Based on the above conditions, the moving distance y of the measured object is: It can be understood that in the embodiment of the present invention, the relative displacement between the measured object and the baseline AB is that the incident light shines on the convex or concave part of the plane of the measured object, and the movement distance of the measured object is the depth of the top of the convex or the bottom of the concave relative to the plane of the measured object. The original three-dimensional point cloud data of the plane to be detected is obtained based on the principle of laser triangulation, and the imaging is stable, not easily affected by ambient light, and has a high detection rate.

[0054] For example, Figure 3a A real battery is shown, and the bottom plane with a depression of the real battery is the plane to be detected. Figure 3b The three-dimensional point cloud data of the bottom of the battery is shown. The three-dimensional point cloud data is obtained based on the principle of laser triangulation. The depression at the bottom of the battery exists in the center of the three-dimensional point cloud data and is not obvious when observed from this perspective. Figure 3c A side view of the three-dimensional point cloud data of the bottom of the battery is shown, and the position of the depression at the bottom of the battery in the three-dimensional point cloud data can be clearly observed.

[0055] S120, determining a target reference plane of the plane to be detected according to the original three-dimensional point cloud data; wherein the number of the inner points in the original three-dimensional point cloud data whose distance to the target reference plane is less than a preset distance threshold is the largest.

[0056] In an embodiment of the present invention, after the original three-dimensional point cloud data of the plane to be detected is obtained, the target reference plane of the plane to be detected can be determined based on the original three-dimensional point cloud data. In space, a plane can be determined by three points. Optionally, the initial reference plane is constructed by randomly selecting three points in the original three-dimensional data, and the inner point corresponding to the initial reference plane is the point in the original three-dimensional point cloud data whose distance to the initial reference plane is less than a preset distance threshold. It can be understood that different initial reference planes correspond to different numbers of inner points. The larger the number of corresponding inner points, the higher the degree of fit to the plane to be detected. Here, the initial reference plane with the largest number of corresponding inner points is selected as the target reference plane. In this way, the target reference plane with the highest degree of fit to the plane to be detected can be determined.

[0057] S130 , reconstructing the original three-dimensional point cloud data based on the target reference plane to generate target three-dimensional point cloud data.

[0058] In an embodiment of the present invention, after determining the target reference plane, the original three-dimensional point cloud data can be reconstructed based on the target reference plane to generate target three-dimensional point cloud data. Optionally, when reconstructing the point cloud of the original three-dimensional point cloud data, common methods include voxel-based reconstruction methods, polyhedron fitting-based reconstruction methods, deep learning-based reconstruction methods, etc. In an embodiment of the present invention, after obtaining the target reference plane, the original three-dimensional point cloud data can be reconstructed through a coordinate transformation matrix. As a result, the plane where the reconstructed target three-dimensional point cloud data is located can coincide with the XY plane in the three-dimensional coordinate system of the space, so as to facilitate subsequent processing.

[0059] For example, Figure 3b As shown in the figure, due to the angle between the battery and the installed 3D forming system, there will be a certain angle between the original 3D point cloud data of the bottom plane of the battery and the spatial 3D coordinate system. Due to the existence of this angle, the Z coordinate values ​​of the 3D point cloud data of different areas of the entire bottom plane of the battery vary greatly, resulting in the inability to effectively segment the defective area based on the Z coordinate value. Therefore, before defect detection, the original 3D point cloud data must be reconstructed based on the target reference plane.

[0060] S140, performing defect detection on the plane to be inspected based on the target three-dimensional point cloud data.

[0061] In an embodiment of the present invention, after the original three-dimensional point cloud data is reconstructed based on the target reference plane, the generated target three-dimensional point cloud data can be used for defect detection of the plane to be detected. Generally speaking, to perform defect detection on the plane to be detected, the following steps can be adopted: 1. Extract the plane. First, it is necessary to extract the plane to be detected, which can be achieved by the plane fitting algorithm in the three-dimensional point cloud data; 2. Determine the defect area. The defect area on the plane to be detected can be divided out by using manual annotation or image segmentation technology; 3. Feature extraction. Extract relevant features from the defect area, such as geometric features, texture features or depth features; 4. Defect detection. Machine learning algorithms, such as support vector machines, neural networks, etc., can be used to train classifiers to classify defect areas and distinguish them from normal areas. At the same time, traditional rule-based methods, such as threshold methods and morphological operations, can also be used; 5. Visualization. Finally, the defect area can be visualized on the three-dimensional point cloud data for subsequent analysis and processing. In an embodiment of the present invention, defect detection is performed based on the reconstructed three-dimensional point cloud data, and the defect area of ​​the plane to be detected can be accurately detected to meet the high-precision detection scenario.

[0062] In the embodiment of the present invention, the original three-dimensional point cloud data of the plane to be detected is obtained, and the microscopic defects that cannot be captured by the human eye can be converted into a data set that can be calculated; the target reference plane of the plane to be detected is determined according to the original three-dimensional point cloud data; wherein, the number of internal points in the original three-dimensional point cloud data whose distance to the target reference plane is less than a preset distance threshold is the largest, and the target reference plane with the highest degree of fit with the plane to be detected can be determined; the original three-dimensional point cloud data is reconstructed based on the target reference plane to generate target three-dimensional point cloud data, and the plane where the reconstructed target three-dimensional point cloud data is located can coincide with the XY plane in the three-dimensional coordinate system of the space, so as to facilitate subsequent processing; the plane to be detected is detected for defects based on the target three-dimensional point cloud data. The technical solution implemented by the present invention can not only solve the technical problem of complex detection background and susceptibility to ambient light when performing defect area detection in 2.5D photometric stereo imaging, but also accurately detect the defect area of ​​the plane to be detected to meet the high-precision detection scenario.

[0063] Embodiment 2

[0064] Figure 4 This is a flow chart of a defect detection method provided by Embodiment 2 of the present invention. The embodiment of the present invention is optimized based on the above embodiment. For solutions not described in detail in the embodiment of the present invention, please refer to the above embodiment. Figure 4 As shown, the method includes:

[0065] S210: Acquire original three-dimensional point cloud data of the plane to be detected.

[0066] S220 , randomly extracting three three-dimensional data points from the original three-dimensional point cloud data, and constructing an initial reference plane of the plane to be detected based on the three three-dimensional data points.

[0067] In the embodiment of the present invention, in order to eliminate the error caused by the defective area, the Ransac plane fitting algorithm is used to filter out the influence of the defective area. Specifically, three three-dimensional data points can be randomly extracted from the original three-dimensional point cloud data, and the initial reference plane of the plane to be detected can be constructed by a formula based on the three three-dimensional data points.

[0068] For example, Figure 3a As shown in the figure, the bottom of the battery is a plane, but there are also some concave areas. If the traditional least squares method is used for plane fitting, the point cloud data of the concave area will also participate in the plane fitting process, thus causing distortion to the fitted plane. In order to eliminate the error caused by the defective area, the Ransac plane fitting algorithm can be used to filter out the influence of the defective area. Figure 3b As shown, the original 3D point cloud data of the battery bottom will have a certain angle with the spatial 3D coordinate system, that is, the component of the normal vector of the plane at the bottom of the battery in the z direction is not 0, so the plane expression can be written as: z = Ax + By + C. Let the original 3D point cloud data be the point set Pi (xi, yi, zi).

[0069] Randomly select three points p1(x1, y1, z1), p2(x2, y2, z2), p3(x3, y3, z3) from the point set Pi and substitute them into z=Ax+By+C to obtain a system of three-variable linear equations:

[0070] z1=A·x1+B·y1+C

[0071] z2=A·x2+B·y2+C

[0072] z3=A·x3+B·y3+C

[0073] By solving the above three-variable linear equation system, we can obtain a set of solutions A1, B1, C1, and determine an initial reference plane z=A1·x+B1·y+C1.

[0074] S230, respectively calculating the distance between each 3D data point in the original 3D point cloud data and the initial reference plane.

[0075] In the embodiment of the present invention, after an initial reference plane is determined, the distances from each 3D data point in the original 3D point cloud data to the initial reference plane can be calculated according to the point-to-plane distance formula. For example, the distance d from all points in the point set Pi to the plane z=A1·x+B1·y+C1 can be calculated according to the point-to-plane distance formula.

[0076] S240. Determine inner points and outer points from the original three-dimensional point cloud data according to distance; wherein the inner points are three-dimensional data points whose distance to the initial reference plane is less than a preset distance threshold, and the outer points are three-dimensional data points whose distance to the initial reference plane is greater than the preset distance threshold.

[0077] In the embodiment of the present invention, a three-dimensional data point whose distance to the initial reference plane is less than a preset distance threshold is called an inlier, and a three-dimensional data point whose distance to the initial reference plane is greater than the preset distance threshold is called an outlier. It is understandable that different initial reference planes correspond to different inliers and outliers, and the preset distance threshold can be set according to the actual application scenario. After determining the distance from each three-dimensional data point in the original three-dimensional point cloud data to the initial reference plane, the inliers and outliers can be determined from the original three-dimensional point cloud data according to the distance.

[0078] Exemplarily, if the distance d from a point in Pi to the plane z=A1·x+B1·y+C1 is less than the threshold value distThr=0.05, the point is defined as an inner point, otherwise it is an outer point, thus the inner points and outer points can be determined from the point set Pi.

[0079] S250, respectively counting the number of inner points and the number of outer points corresponding to the initial reference plane, and calculating the ratio of the number of inner points to the number of outer points.

[0080] In an embodiment of the present invention, after determining the inliers and outliers from the original three-dimensional point cloud data according to the distance, the number of inliers and outliers corresponding to the initial reference plane can be counted respectively, and the ratio of the number of inliers to the number of outliers can be calculated. It can be understood that the larger the ratio of the number of inliers to the number of outliers, the more inliers the initial reference plane corresponds to, which means that the initial reference plane has a better fitting quality for the plane to be detected. Optionally, when the ratio of the number of inliers to the number of outliers is greater than a preset value, the inliers and outliers corresponding to the initial reference plane are stored in a container.

[0081] For example, after determining the inner points and outer points from the point set Pi, the number of inner points and the number of outer points corresponding to the plane z = A1 x + B1 y + C1 can be counted respectively, and the ratio of the number of inner points to the number of outer points can be calculated. The ratio r of the number of inner points to the number of outer points is calculated, and if r is greater than the threshold ratioThr = 1, the inner points and outer points corresponding to the initial reference plane are saved in a container.

[0082] S260, repeat the above process N times, and select the initial reference plane with the largest ratio of the number of inner points to the number of outer points from the N constructed initial reference planes as the target reference plane.

[0083] In an embodiment of the present invention, the plane to be detected can be fitted multiple times by constructing the initial reference plane multiple times, so as to select the initial reference plane with the largest ratio of the number of inner points to the number of outer points as the target reference plane. Specifically, the above steps S220-S250 are repeated N times, and the initial reference plane with the largest ratio of the number of inner points to the number of outer points is selected from the N constructed initial reference planes as the target reference plane. Optionally, after repeating N times, the inner point and the outer point with the largest ratio of the number of inner points to the number of outer points in the container are selected, and the initial reference plane corresponding to the inner point and the outer point is selected as the target reference plane. In this way, the target reference plane with the highest degree of fit with the plane to be detected can be determined.

[0084] Exemplarily, the process of randomly selecting three points from the point set Pi, constructing an initial reference plane, determining the inner points and outer points corresponding to the initial reference plane, counting the number of inner points and the number of outer points corresponding to the initial reference plane, calculating the ratio of the number of inner points to the number of outer points, and saving the inner points and outer points with a ratio greater than 1 into a container is repeated N times. The inner point and outer point with the largest ratio of the number of inner points to the number of outer points in the container are selected, and the initial reference plane corresponding to the inner point and the outer point is selected as the target reference plane.

[0085] S270, constructing a coordinate transformation matrix based on the target reference plane.

[0086] In the embodiment of the present invention, after obtaining the target reference plane, a coordinate transformation matrix can be constructed according to the target reference plane. The target reference plane is z=Ax+By+C, and the coordinate transformation matrix is ​​used to reconstruct the original three-dimensional point cloud data. Specifically, after obtaining the target reference plane z=Ax+By+C, first solve the normalized normal vector z=Ax+By+C. in, After obtaining the normalized normal vector z=Ax+By+C Afterwards, the coordinate transformation matrix can be constructed based on the normal vector.

[0087] S280, reconstructing the original three-dimensional point cloud data according to the coordinate transformation matrix to generate target three-dimensional point cloud data.

[0088] In the embodiment of the present invention, after constructing a coordinate transformation matrix based on the target reference plane, point cloud reconstruction can be performed on the original three-dimensional point cloud data according to the coordinate transformation matrix to generate target three-dimensional point cloud data. Specifically, point cloud reconstruction is performed on the original three-dimensional point cloud data according to the coordinate transformation matrix to generate target three-dimensional point cloud data, including:

[0089] The original 3D point cloud data is reconstructed according to the following formula to generate the target 3D point cloud data:

[0090]

[0091] Where F(Fx, Fy, Fz) is any point in the original 3D point cloud data, and S(Sx, Sy, Sz) is the point reconstructed from F(Fx, Fy, Fz) in the target 3D point cloud data.

[0092]

[0093] Thus, the plane where the reconstructed target three-dimensional point cloud data is located can coincide with the XY plane in the three-dimensional spatial coordinate system to facilitate subsequent processing. Figure 5a The reconstructed 3D point cloud data of the bottom of the battery is shown. Figure 5b A side view of the reconstructed 3D point cloud data of the bottom of the battery is shown.

[0094] S290. Determine the defect area of ​​the plane to be inspected and the defect feature information corresponding to the defect area based on the target three-dimensional point cloud data; wherein the defect feature information includes the depth, length, width and area of ​​the defect area.

[0095] In the embodiment of the present invention, after the original three-dimensional point cloud data is reconstructed according to the coordinate transformation matrix, the plane where the generated target three-dimensional point cloud data is located coincides with the XY plane of the spatial three-dimensional coordinate system, and the Z value of the three-dimensional point cloud data reflects the height of the plane to be detected. According to steps S220-S250, the plane to be detected is refitted, and the target three-dimensional point cloud data is segmented at the preset depth threshold above and below the reference plane with the fitted plane to be detected as the reference, so as to determine the defect area of ​​the plane to be detected based on the target three-dimensional point cloud data.

[0096] Specifically, determining the defect area of ​​the plane to be inspected based on the target three-dimensional point cloud data includes:

[0097] Determine an area formed by three-dimensional point cloud data having a depth value greater than a first preset depth threshold in the target three-dimensional point cloud data as a convex defect area in the plane to be detected;

[0098] An area formed by three-dimensional point cloud data having a depth value less than a second preset depth threshold in the target three-dimensional point cloud data is determined as a concave defect area in the plane to be detected.

[0099] In the embodiment of the present invention, when the depth value of the three-dimensional point cloud data, i.e., the Z value, is greater than the first preset depth threshold, it indicates that the three-dimensional point cloud data is located at a convex position in the target three-dimensional point cloud data, and the area formed by the three-dimensional point cloud data of this type is determined as a convex defect area in the plane to be detected. When the depth value of the three-dimensional point cloud data, i.e., the Z value, is less than the second preset depth threshold, it indicates that the three-dimensional point cloud data is located at a concave position in the target three-dimensional point cloud data, and the area formed by the three-dimensional point cloud data of this type is determined as a concave defect area in the plane to be detected.

[0100] For example, Figure 6 A schematic diagram of three-dimensional point cloud data segmentation based on the Z value is shown. The reference plane in the figure is the bottom plane of the battery after re-fitting, h is the preset depth threshold, the part with a depth value greater than h is a raised defect area, and the part with a depth value less than h is a sunken defect area.

[0101] In the implementation of the present invention, after determining the defect area of ​​the plane to be detected, the defect feature information corresponding to the defect area can be measured, wherein the defect feature information includes the depth, length, width and area of ​​the defect area. Specifically, the target three-dimensional point cloud data area of ​​the segmented defect area is converted to an 8-bit grayscale image, and the traditional 2D image processing tools are used to locate and measure the length, width and area of ​​the defect area. The position information of the defect area in the grayscale image is mapped to the target three-dimensional point cloud data of the segmented defect area, and the Z value of the corresponding target three-dimensional point cloud data is counted to measure the depth value of the defect area. In this way, the length, width, area and depth values ​​of the defect area can be accurately measured with high precision.

[0102] For example, Figure 7a The target 3D point cloud data of the segmented concave defect area is shown. Figure 7b A side view of the target 3D point cloud data of the segmented concave defect area is shown. Figure 8 A flow chart for measuring defective areas is shown. As shown in the figure, the target three-dimensional data of the defective area on the bottom plane of the battery is first converted into an 8-bit grayscale image. The spots corresponding to the defective area in the 8-bit grayscale image are automatically located by the spot tool, and the length and area of ​​the defective area are measured. At the same time, the depth value is obtained by mapping the position information of the spots to the target three-dimensional point cloud data of the corresponding defective area. Finally, the controlled length, width, area, and depth values ​​are compared to see whether the produced batteries meet the standards. Figure 9a The target 3D point cloud data of the sunken defect area at the bottom of the battery after segmentation is shown. Figure 9b An 8-bit grayscale image of the defect area is shown.

[0103] In an embodiment of the present invention, original 3D point cloud data of a plane to be inspected is obtained, and microscopic defects that cannot be captured by the human eye can be converted into a data set that can be operated on; three 3D data points are randomly extracted from the original 3D point cloud data, and an initial reference plane of the plane to be inspected is constructed based on the three 3D data points; the distance from each 3D data point in the original 3D point cloud data to the initial reference plane is calculated respectively; the inner point and the outer point are determined from the original 3D point cloud data based on the distance; wherein the inner point is a 3D data point whose distance to the initial reference plane is less than a preset distance threshold, and the outer point is a 3D data point whose distance to the initial reference plane is greater than the preset distance threshold; the number of inner points and the number of outer points corresponding to the initial reference plane are counted respectively, and the ratio of the number of inner points to the number of outer points is calculated; and the above process is repeated N times , select the initial reference plane with the largest ratio of the number of internal points to the number of external points from the constructed N initial reference planes as the target reference plane, and determine the target reference plane with the highest degree of fit with the plane to be detected; construct a coordinate transformation matrix based on the target reference plane; reconstruct the original three-dimensional point cloud data according to the coordinate transformation matrix to generate target three-dimensional point cloud data, and the plane where the reconstructed target three-dimensional point cloud data is located can coincide with the XY plane in the three-dimensional coordinate system of the space, so as to facilitate subsequent processing; determine the defect area of ​​the plane to be detected and the defect feature information corresponding to the defect area based on the target three-dimensional point cloud data; wherein the defect feature information includes the depth, length, width and area of ​​the defect area, and the length, width, area and depth values ​​of the defect area can be accurately measured with high precision. Through the technical solution implemented by the present invention, various information of the defect area can be accurately and quantitatively measured to meet the high-precision detection scenario.

[0104] Embodiment 3

[0105] Fig.10 This is a schematic diagram of the structure of a defect detection device provided in Embodiment 3 of the present invention. Fig.10 As shown, the device comprises:

[0106] The three-dimensional point cloud data acquisition module 310 is used to acquire the original three-dimensional point cloud data of the plane to be detected;

[0107] A target reference plane determination module 320 is used to determine the target reference plane of the plane to be detected according to the original three-dimensional point cloud data; wherein the number of the inner points in the original three-dimensional point cloud data whose distance to the target reference plane is less than a preset distance threshold is the largest;

[0108] A three-dimensional point cloud data reconstruction module 330 is used to reconstruct the original three-dimensional point cloud data based on the target reference plane to generate target three-dimensional point cloud data;

[0109] The defect detection module 340 is used to perform defect detection on the plane to be detected based on the target three-dimensional point cloud data.

[0110] In the embodiment of the present invention, the target reference plane determination module 320 includes:

[0111] An initial reference plane construction unit, used for randomly extracting three three-dimensional data points from the original three-dimensional point cloud data, and constructing an initial reference plane of the plane to be detected according to the three three-dimensional data points;

[0112] A distance calculation unit, used to calculate the distance between each three-dimensional data point in the original three-dimensional point cloud data and the initial reference plane;

[0113] an inner and outer point determination unit, configured to determine inner points and outer points from the original three-dimensional point cloud data according to the distance; wherein the inner point is a three-dimensional data point whose distance to the initial reference plane is less than the preset distance threshold, and the outer point is a three-dimensional data point whose distance to the initial reference plane is greater than the preset distance threshold;

[0114] a ratio calculation unit, configured to respectively count the number of inner points and the number of outer points corresponding to the initial reference plane, and calculate the ratio of the number of inner points to the number of outer points;

[0115] The target reference plane determination unit is used to repeat the above process N times, and select the initial reference plane with the largest ratio of the number of inner points to the number of outer points from the constructed N initial reference planes as the target reference plane.

[0116] In the embodiment of the present invention, the 3D point cloud data reconstruction module 330 includes:

[0117] A coordinate conversion matrix construction unit, used to construct a coordinate conversion matrix based on the target reference plane;

[0118] The target three-dimensional point cloud data generating unit is used to perform point cloud reconstruction on the original three-dimensional point cloud data according to the coordinate transformation matrix to generate target three-dimensional point cloud data.

[0119] Optionally, the target reference plane is z=Ax+By+C, and the target three-dimensional point cloud data generating unit is specifically used for:

[0120] The original three-dimensional point cloud data is reconstructed according to the following formula to generate target three-dimensional point cloud data:

[0121]

[0122] Wherein, F(Fx, Fy, Fz) is any point in the original three-dimensional point cloud data, and S(Sx, Sy, Sz) is a point in the target three-dimensional point cloud data after reconstructing F(Fx, Fy, Fz);

[0123]

[0124] In the embodiment of the present invention, the defect detection module 340 includes:

[0125] A defect feature information determination unit is used to determine the defect area of ​​the plane to be detected and the defect feature information corresponding to the defect area based on the target three-dimensional point cloud data; wherein the defect feature information includes the depth, length, width and area of ​​the defect area.

[0126] Optionally, the defect characteristic information determining unit includes:

[0127] A convex defect area determination subunit is used to determine an area consisting of three-dimensional point cloud data having a depth value greater than a first preset depth threshold in the target three-dimensional point cloud data as a convex defect area in the plane to be detected;

[0128] The concave defect area determination subunit is used to determine the area composed of the three-dimensional point cloud data with a depth value less than a second preset depth threshold in the target three-dimensional point cloud data as the concave defect area in the plane to be detected.

[0129] In the embodiment of the present invention, the 3D point cloud data acquisition module 310 includes:

[0130] The three-dimensional point cloud data acquisition unit is used to acquire the original three-dimensional point cloud data of the plane to be detected based on the laser triangulation principle.

[0131] The defect detection device provided in the embodiment of the present invention can execute the defect detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0132] Embodiment 4

[0133] Fig.11 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0134] like Fig.11As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0135] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0136] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a defect detection method.

[0137] In some embodiments, the defect detection method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the defect detection method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the defect detection method in any other appropriate manner (e.g., by means of firmware).

[0138] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0139] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0140] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0141] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0142] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0143] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0144] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0145] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A defect detection method, characterized in that: include: Obtaining original three-dimensional point cloud data of the plane to be detected; Determining a target reference plane of the plane to be detected according to the original three-dimensional point cloud data; wherein the number of inliers in the original three-dimensional point cloud data whose distance to the target reference plane is less than a preset distance threshold is the largest; Performing point cloud reconstruction on the original three-dimensional point cloud data based on the target reference plane to generate target three-dimensional point cloud data; wherein the point cloud reconstruction includes: a voxel-based reconstruction method, a polyhedron fitting-based reconstruction method, a deep learning-based reconstruction method, or reconstructing the original three-dimensional point cloud data through a coordinate transformation matrix; Defect detection is performed on the plane to be detected based on the target three-dimensional point cloud data.

2. The method according to claim 1, characterized in that Determining the target reference plane of the plane to be detected according to the original three-dimensional point cloud data includes: Randomly extracting three three-dimensional data points from the original three-dimensional point cloud data, and constructing an initial reference plane of the plane to be detected according to the three three-dimensional data points; Calculating the distance between each three-dimensional data point in the original three-dimensional point cloud data and the initial reference plane respectively; Determine an inner point and an outer point from the original three-dimensional point cloud data according to the distance; wherein the inner point is a three-dimensional data point whose distance to the initial reference plane is less than the preset distance threshold, and the outer point is a three-dimensional data point whose distance to the initial reference plane is greater than the preset distance threshold; Respectively counting the number of inner points and the number of outer points corresponding to the initial reference plane, and calculating the ratio of the number of inner points to the number of outer points; Repeat the above process N times, and select the initial reference plane with the largest ratio of the number of inner points to the number of outer points from the N constructed initial reference planes as the target reference plane.

3. The method according to claim 1, characterized in that Performing point cloud reconstruction on the original three-dimensional point cloud data to generate target three-dimensional point cloud data includes: Constructing a coordinate transformation matrix based on the target reference plane; The original three-dimensional point cloud data is reconstructed according to the coordinate transformation matrix to generate target three-dimensional point cloud data.

4. The method according to claim 3, characterized in that The target reference plane is z=Ax+By+C, Reconstructing the original three-dimensional point cloud data according to the coordinate transformation matrix to generate target three-dimensional point cloud data includes: The original three-dimensional point cloud data is reconstructed according to the following formula to generate target three-dimensional point cloud data: ； Wherein, F(Fx, Fy, Fz) is any point in the original three-dimensional point cloud data, and S(Sx, Sy, Sz) is a point in the target three-dimensional point cloud data after reconstructing F(Fx, Fy, Fz); ， ， ， 。 5. The method according to claim 1, characterized in that Performing defect detection on the plane to be detected based on the target three-dimensional point cloud data includes: Based on the target three-dimensional point cloud data, the defect area of ​​the plane to be detected and the defect feature information corresponding to the defect area are determined; wherein the defect feature information includes the depth, length, width and area of ​​the defect area.

6. The method according to claim 5, characterized in that Determining the defect area of ​​the plane to be detected based on the target three-dimensional point cloud data includes: Determine an area formed by three-dimensional point cloud data having a depth value greater than a first preset depth threshold in the target three-dimensional point cloud data as a convex defect area in the plane to be detected; An area formed by three-dimensional point cloud data having a depth value less than a second preset depth threshold in the target three-dimensional point cloud data is determined as a concave defect area in the plane to be detected.

7. The method according to any one of claims 1 to 6, characterized in that: Get the original 3D point cloud data of the plane to be detected, including: The original three-dimensional point cloud data of the plane to be detected is obtained based on the principle of laser triangulation.

8. A defect detection device, characterized in that: include: A three-dimensional point cloud data acquisition module is used to acquire original three-dimensional point cloud data of the plane to be detected; A target reference plane determination module is used to determine the target reference plane of the plane to be detected according to the original three-dimensional point cloud data; wherein the number of the inner points in the original three-dimensional point cloud data whose distance to the target reference plane is less than a preset distance threshold is the largest; A three-dimensional point cloud data reconstruction module, used to perform point cloud reconstruction on the original three-dimensional point cloud data based on the target reference plane to generate target three-dimensional point cloud data; wherein the point cloud reconstruction includes: a voxel-based reconstruction method, a polyhedron fitting-based reconstruction method, a deep learning-based reconstruction method, or reconstruction of the original three-dimensional point cloud data through a coordinate transformation matrix; A defect detection module is used to perform defect detection on the plane to be detected based on the target three-dimensional point cloud data.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the defect detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the defect detection method according to any one of claims 1 to 7 when executed.

Images