Lock attaching state detection method, device and equipment based on laser contourgraph and storage medium

Abstract

The invention discloses a locking state detection method, device and equipment based on a laser contourgraph and a storage medium, and relates to the technical field of visual inspection, and the locking state detection method based on the laser contourgraph comprises the steps: scanning a locking product through the laser contourgraph, and obtaining target three-dimensional point cloud data; clustering and screening the target three-dimensional point cloud data to obtain a reference plane initial point cloud and a screw locking initial point cloud; performing plane fitting on the reference plane initial point cloud to obtain a reference plane; clustering, screening and fitting the screw locking initial point cloud to obtain a measurement plane set; and determining the locking state of the locking product through the measurement plane set and the reference plane. According to the invention, the state after locking can be detected through laser scanning, and the accuracy of locking state detection is improved.

Description

Technical Field

[0001] The present application relates to the technical field of visual inspection, and particularly relates to a method, device, equipment and storage medium for detecting the locking state based on a laser profiler. Background Technique

[0002] In the automated assembly process, there is a common locking process, that is, screwing a screw into a hole according to a specified torque. Due to differences in incoming materials or environmental factors, it cannot be guaranteed that the locking is complete. Therefore, an additional inspection process needs to be added after the locking to detect the locking state. Currently, the most common method in the industry is manual inspection, which requires labor and the inspection results vary from person to person, and it is impossible to ensure accuracy and stability; or using a camera to take pictures of the locked state for inspection, but it is difficult to obtain height information by camera shooting, and it is difficult to judge the situation where the screw floats, that is, the screw is not completely locked in place. Therefore, how to improve the accuracy of the locking state detection is still a problem to be solved.

[0003] The above content is only used to assist in understanding the technical solution of the present application, and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of the present application is to provide a method, device, equipment and storage medium for detecting the locking state based on a laser profiler, aiming to solve the technical problem of how to improve the accuracy of the locking state detection.

[0005] To achieve the above object, the present application proposes a method for detecting the locking state based on a laser profiler, and the method includes: scanning a locked product with a laser profiler to obtain target three-dimensional point cloud data; clustering and screening the target three-dimensional point cloud data to obtain an initial point cloud of the reference plane and an initial point cloud of the screw locking; fitting the initial point cloud of the reference plane to obtain a reference plane; clustering, screening and fitting the initial point cloud of the screw locking to obtain a set of measurement planes; determining the locking state of the locked product through the set of measurement planes and the reference plane.

[0006] In one embodiment, the step of scanning the locked product with a laser profiler to obtain target three-dimensional point cloud data includes:

[0007] Scanning the locked product with a laser profiler to obtain the original three-dimensional point cloud data of the locked product;

[0008] Obtaining a preset three-dimensional coordinate range, and filtering the original three-dimensional point cloud data through the preset three-dimensional coordinate range to obtain intermediate three-dimensional point cloud data;

[0009] Obtain the average distance between each point in the intermediate three-dimensional point cloud data and its surrounding neighboring points, and calculate the mean and standard deviation of the average distance;

[0010] Screen the intermediate three-dimensional point cloud data through the mean and standard deviation to obtain the target three-dimensional point cloud data.

[0011] In one embodiment, the step of clustering and screening the target three-dimensional point cloud data to obtain the initial point cloud of the reference plane and the initial point cloud of screw attachment includes:

[0012] Calculate the spatial normal vector of each point in the target three-dimensional point cloud data to obtain a set of normal vectors;

[0013] Obtain a preset normal vector angle threshold, and cluster the target three-dimensional point cloud data through the set of normal vectors and the preset normal vector angle threshold to obtain a set of point cloud clustering data;

[0014] Screen out the initial point cloud of the reference plane and the initial point cloud of screw attachment from the set of point cloud clustering data according to a preset point cloud size.

[0015] In one embodiment, the step of calculating the spatial normal vector of each point in the target three-dimensional point cloud data to obtain a set of normal vectors includes:

[0016] Obtain the set of neighboring points of each point in the target three-dimensional point cloud data;

[0017] Calculate the local covariance matrix through the set of neighboring points and perform eigenvalue decomposition to obtain eigenvectors;

[0018] Obtain a set of normal vectors according to the eigenvectors.

[0019] In one embodiment, the step of fitting the initial point cloud of the reference plane to obtain the reference plane includes:

[0020] Calculate the center point coordinates of the initial point cloud of the reference plane;

[0021] Subtract the center point coordinates from the coordinates of each point in the initial point cloud of the reference plane to obtain the target coordinates;

[0022] Determine the plane direction through the target coordinates;

[0023] Obtain the reference plane according to the center point coordinates and the plane direction.

[0024] In one embodiment, the step of clustering, screening, and fitting the initial point cloud of screw attachment to obtain a set of measurement planes includes:

[0025] Calculate the spatial normal vectors of each point in the initial point cloud of the screw attachment to obtain a set of screw point cloud normal vectors;

[0026] Obtain a preset screw point cloud normal vector angle threshold and a preset distance threshold;

[0027] Cluster according to the set of screw point cloud normal vectors and the preset screw point cloud normal vector angle, and the distance between any two points in the initial point cloud of the screw attachment and the preset distance threshold to obtain a set of screw point cloud clustering data;

[0028] Filter the set of screw point cloud clustering data according to the preset screw point cloud size and the preset screw point cloud shape, and fit the filtering result to obtain a set of measurement planes.

[0029] In one embodiment, the step of determining the attachment state of the attached product by using the set of measurement planes and the reference plane includes:

[0030] Obtain the number of measurement planes according to the set of measurement planes;

[0031] Obtain the angle and distance between each measurement plane and the reference plane according to the set of measurement planes and the reference plane;

[0032] Determine the attachment state of the attached product according to the number of measurement planes, the angle, and the distance.

[0033] In addition, to achieve the above object, the present application also proposes a lock attachment state detection device based on a laser profiler. The lock attachment state detection device based on a laser profiler includes:

[0034] A scanning module for scanning an attached product through a laser profiler to obtain target three-dimensional point cloud data;

[0035] A classification module for clustering and filtering the target three-dimensional point cloud data to obtain an initial point cloud of the reference plane and an initial point cloud of the screw attachment;

[0036] A fitting module for fitting the initial point cloud of the reference plane to obtain a reference plane;

[0037] A filtering module for clustering, filtering, and fitting the initial point cloud of the screw attachment to obtain a set of measurement planes;

[0038] A determination module for determining the attachment state of the attached product by using the set of measurement planes and the reference plane.

[0039] In addition, to achieve the above object, the present application further provides a locking state detection device based on a laser profiler, the device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the locking state detection method based on a laser profiler as described above.

[0040] In addition, to achieve the above object, the present application further provides a storage medium, the storage medium being a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the locking state detection method based on a laser profiler as described above.

[0041] In addition, to achieve the above object, the present application further provides a computer program product, the computer program product comprising a computer program, and when the computer program is executed by a processor, it implements the steps of the locking state detection method based on a laser profiler as described above.

[0042] The present application provides a locking state detection method based on a laser profiler. The present application scans the locked product with a laser profiler to obtain target three-dimensional point cloud data; clusters and filters the target three-dimensional point cloud data to obtain an initial point cloud of a reference plane and an initial point cloud of screw locking; performs plane fitting on the initial point cloud of the reference plane to obtain a reference plane; performs clustering, filtering, and fitting on the initial point cloud of screw locking to obtain a set of measurement planes; and determines the locking state of the locked product through the set of measurement planes and the reference plane.

[0043] In summary, by using a laser profiler to scan the locked product, the present application can accurately and quickly measure the three-dimensional contour and dimensions of the locked product, obtain target three-dimensional point cloud data, and perform locking state detection on the target three-dimensional point cloud data, thereby improving the accuracy of locking state detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0045] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0046] Figure 1 It is a schematic flowchart provided for Embodiment 1 of the locking state detection method based on a laser profiler of the present application;

[0047] Figure 2 Schematic diagram of the working device of the laser profiler provided in the first embodiment of the locking state detection method based on the laser profiler of the present application;

[0048] Figure 3 Schematic diagram of the reference plane provided in the first embodiment of the locking state detection method based on the laser profiler of the present application;

[0049] Figure 4 Schematic diagram of the measurement plane provided in the first embodiment of the locking state detection method based on the laser profiler of the present application;

[0050] Figure 5 Flow chart provided in the second embodiment of the locking state detection method based on the laser profiler of the present application;

[0051] Figure 6 Brief flow chart of the locking state detection method based on the laser profiler provided in the second embodiment of the present application;

[0052] Figure 7 Schematic diagram of the module structure of the locking state detection device based on the laser profiler in the embodiment of the present application;

[0053] Figure 8 Schematic diagram of the device structure of the hardware operating environment involved in the locking state detection method based on the laser profiler in the embodiment of the present application.

[0054] The realization of the purpose, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. Detailed implementation manners

[0055] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0056] In order to better understand the technical solutions of the present application, the following will be described in detail with reference to the accompanying drawings of the specification and the specific implementation manners.

[0057] The main solution of the present application is to scan the locked product by a laser profiler to obtain target three-dimensional point cloud data; perform clustering and screening on the target three-dimensional point cloud data to obtain the initial point cloud of the reference plane and the initial point cloud of screw locking; perform plane fitting on the initial point cloud of the reference plane to obtain the reference plane; perform clustering, screening and fitting on the initial point cloud of screw locking to obtain a set of measurement planes; determine the locking state of the locked product through the set of measurement planes and the reference plane.

[0058] Currently, the most common method in the industry is manual inspection, which requires a lot of manpower. At the same time, the inspection results vary from person to person, and it is impossible to guarantee accuracy and stability. Or, a camera is used to take pictures of the locked state for inspection. However, since it is difficult to obtain height information from camera shooting, it is very difficult to judge the situation where the screw floats, that is, the screw is not fully locked in place. Therefore, how to improve the accuracy of the locked state detection is still a problem to be solved.

[0059] In this application, by using a laser profiler to scan the locked product, the three-dimensional contour and size of the locked product can be accurately and quickly measured, and the target three-dimensional point cloud data can be obtained. Then, the locked state is detected based on the target three-dimensional point cloud data, which improves the accuracy of the locked state detection.

[0060] Based on this, the embodiment of this application provides a locked state detection method based on a laser profiler. Refer to Figure 1 , Figure 1 which is a schematic flowchart of the first embodiment of the locked state detection method based on a laser profiler in this application.

[0061] In this embodiment, the locked state detection method based on a laser profiler includes steps S10 to S50:

[0062] Step S10: Scan the locked product with a laser profiler to obtain target three-dimensional point cloud data;

[0063] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device that can implement the above functions, a locked state detection device based on a laser profiler, etc. Hereinafter, a locked state detection device based on a laser profiler will be used as an example to illustrate this embodiment and the following embodiments.

[0064] It should be noted that a laser profiler is a device that uses laser scanning technology to accurately and quickly measure the three-dimensional contour and size of an object. Refer to Figure 2 , Figure 2 which is a schematic diagram of the working device of a laser profiler. In the figure, 1 is the overall frame of the device, which is used to maintain the overall stability of the device; 2 is the working area, which is set in the overall frame of the device; 3 is the motion mechanism, which includes three degrees of freedom of motion; 4 is the laser profiler, which is installed on the motion mechanism 3 to cooperate with the motion mechanism to scan and image and generate a 3D map, and upload and analyze the generated 3D map.

[0065] It is understandable that first, the locked product is placed on the area to be detected, and the surface to be detected is facing upward so that it can be scanned by the profiler. Then, the moving mechanism drives the laser profiler to scan along a pre-set path, and the scanning range covers the area to be detected on the product. Moreover, in order to improve the scanning accuracy and reduce the influence of unstable speed on the imaging effect, an encoder can be added to the moving mechanism and the encoder signal can be connected to the laser profiler as a trigger signal to improve the accuracy of the relationship between the photographing position and the actual position.

[0066] In a feasible way, the step of scanning the locked product by the laser profiler to obtain the target three-dimensional point cloud data includes:

[0067] Scanning the locked product by the laser profiler to obtain the original three-dimensional point cloud data of the locked product;

[0068] Obtaining a preset three-dimensional coordinate range, and filtering the original three-dimensional point cloud data through the preset three-dimensional coordinate range to obtain intermediate three-dimensional point cloud data;

[0069] Obtaining the average distance between each point in the intermediate three-dimensional point cloud data and its surrounding neighboring points, and calculating the mean and standard deviation of the average distance;

[0070] Filtering the intermediate three-dimensional point cloud data through the mean and standard deviation to obtain the target three-dimensional point cloud data.

[0071] It is understandable that after scanning the locked product by the laser profiler to obtain the original three-dimensional point cloud data of the locked product, the original three-dimensional point cloud data can be filtered through a preset three-dimensional coordinate range to obtain intermediate three-dimensional point cloud data, so that each point in the intermediate three-dimensional point cloud data satisfies the following relationship:

[0072]

[0073] Among them, P1.x, P1.y, and P1.z respectively represent the three-dimensional coordinates of each point, and x min , x max , y min , y max , z min , z max represent the preset three-dimensional coordinate range. In the case where the position differences of the locked products are relatively large, the three-dimensional coordinate range can also be dynamically calculated according to the characteristic coordinates of the point cloud obtained each time. Specifically, the center point Pc of the point cloud corresponding to the product features is obtained through feature comparison, and the maximum and minimum values of x, y, and z of the area to be filtered are calculated through the preset maximum and minimum offsets of x, y, and z:

[0074]

[0075] Among them, x min , x max , y min , y max , z min , z max represent the three-dimensional coordinate range, P c .x, P c .y, P c .z represents the three-dimensional coordinates of the cloud center point Pc, x min_o , x max_o , y min_o , y max_o , z min_o , z max_o represent the preset maximum and minimum offsets of x, y, and z.

[0076] It can be understood that after obtaining the intermediate three-dimensional point cloud data, the distance average value of each point in the intermediate three-dimensional point cloud data can also be statistically calculated based on the average value of the distances from the n1 closest points, where n1 is a preset quantity, so as to obtain the mean value μ and standard deviation σ of the distance average values corresponding to all points. By statistical screening, a subset of the intermediate three-dimensional point cloud data is obtained, that is, the target three-dimensional point cloud data, such that the corresponding distance average value of each point P2 in the target three-dimensional point cloud data satisfies the following relationship:

[0077] μ - 3σ <= dis(P2) <= μ + 3σ

[0078] Step S20: Cluster and screen the target three-dimensional point cloud data to obtain the initial point cloud of the reference plane and the initial point cloud of screw attachment;

[0079] It can be understood that after obtaining the target three-dimensional point cloud data, the reference plane point cloud and the screw point cloud also need to be screened out. Therefore, through clustering and screening, the initial point cloud of the reference plane and the initial point cloud of screw attachment can be obtained first.

[0080] Step S30: Fit the initial point cloud of the reference plane to obtain the reference plane;

[0081] It can be understood that after obtaining the initial point cloud of the reference plane, the reference plane can be obtained by fitting.

[0082] In a feasible manner, the step of fitting the initial point cloud of the reference plane to obtain the reference plane includes:

[0083] Calculate the center point coordinates of the initial point cloud of the reference plane;

[0084] Subtract the center point coordinates from the coordinates of each point in the initial point cloud of the reference plane to obtain the target coordinates;

[0085] Determine the plane direction based on the target coordinates;

[0086] Obtain a reference plane based on the center point coordinates and the plane direction.

[0087] It should be noted that the reference plane can be understood as an ideal reference plane induced from a set of three-dimensional points, which is used to describe the overall spatial distribution characteristics of these points. Refer to Figure 3 , Figure 3 is a schematic diagram of the reference plane, Figure 3 The bottom plane point cloud is the reference plane point cloud, and the raised part point cloud is the screw attachment point cloud.

[0088] It can be understood that first, it is necessary to calculate the average coordinate values of each point in the reference plane initial point cloud in the x, y, and z directions to obtain a virtual center point coordinate. To eliminate position interference, then subtract the center point coordinate from the coordinate of each point in the reference plane initial point cloud to obtain the target coordinate. By analyzing the target coordinate, determine the plane direction, and the plane direction needs to meet the condition that when the plane passes through the center point in this direction, the overall vertical distance from all points to the plane is the smallest. Finally, the reference plane can be obtained through the determined plane direction and the center point position.

[0089] Step S40: Cluster, screen, and fit the screw attachment initial point cloud to obtain a set of measurement planes;

[0090] It can be understood that the screw attachment initial point cloud also needs to be processed to obtain a measurement plane, and since multiple attachments are usually detected simultaneously during the attachment state detection, a set of measurement planes will be obtained. Refer to Figure 4 , Figure 4 is a schematic diagram of the measurement plane, Figure 4 The bottom in it is the reference plane, and the four planes above the reference plane are the measurement planes.

[0091] In a feasible manner, the step of clustering, screening, and fitting the screw attachment initial point cloud to obtain a set of measurement planes includes:

[0092] Calculate the spatial normal vector of each point in the screw attachment initial point cloud to obtain a set of screw point cloud normal vectors;

[0093] Obtain a preset screw point cloud normal vector angle threshold and a preset distance threshold;

[0094] Cluster according to the set of screw point cloud normal vectors and the preset screw point cloud normal vector angle, and the distance between any two points in the screw attachment initial point cloud and the preset distance threshold to obtain a set of screw point cloud clustering data;

[0095] Filter the screw point cloud clustering data set according to the preset screw point cloud size and the preset screw point cloud shape, and perform fitting on the filtering result to obtain a set of measurement planes.

[0096] It can be understood that by calculating the spatial normal vector of each point in the initial screw attachment point cloud, and then clustering according to the angle similarity and distance between the normal vector of each point and the preset screw point cloud normal vector, so that the angle difference between the normal vectors of any two points in the calculated cluster is not greater than the preset screw point cloud normal vector angle, and the distance between any two points is not greater than the preset distance threshold. Screen according to the classified point cloud size and shape, and screen out the point cloud that meets the conditions, which is the screw head point cloud to be detected; then for each screw head point cloud, cluster again according to the angle similarity and distance of the normal vector, screen according to the classified point cloud size, and screen out the point cloud that most conforms to the characteristic size, and then use the least squares method to fit to obtain a set of measurement planes. The screw head in this embodiment is a plane. In actual situations, if the entire screw head is a spherical surface or other curved surface, the curved surface can also be fitted by the least squares method for calculation, and this embodiment does not limit this.

[0097] Step S50: Determine the attachment state of the attached product through the set of measurement planes and the reference plane.

[0098] It can be understood that the measurement plane is the screw head plane, and the attachment state can be detected by the angle and distance between the measurement plane and the reference plane.

[0099] In a feasible manner, the step of determining the attachment state of the attached product through the set of measurement planes and the reference plane includes:

[0100] Obtain the number of measurement planes according to the set of measurement planes;

[0101] Obtain the angle and distance between each measurement plane and the reference plane according to the set of measurement planes and the reference plane;

[0102] Determine the attachment state of the attached product according to the number of measurement planes, the angle, and the distance.

[0103] It can be understood that first, it is judged whether the number of measurement planes is consistent with the number of screws of the attached product. If not, the attachment state can be considered unqualified; if so, then detect according to the angle and distance between the measurement plane and the reference plane, calculate the angle and distance between each measurement plane and the reference plane respectively, and judge whether the corresponding angle and distance are within the qualified interval range according to the preset range value. If any one is not within the range, it is judged that the screw attachment is unqualified.

[0104] This embodiment provides a method for detecting the locking state based on a laser profiler. In this application, the laser profiler is used to scan the locked product to obtain the target three-dimensional point cloud data; the target three-dimensional point cloud data is clustered and screened to obtain the initial point cloud of the reference plane and the initial point cloud of the screw locking; the initial point cloud of the reference plane is plane-fitted to obtain the reference plane; the initial point cloud of the screw locking is clustered, screened, and fitted to obtain a set of measurement planes; the locking state of the locked product is determined by the set of measurement planes and the reference plane.

[0105] In summary, in this embodiment, by using a laser profiler to scan the locked product, the three-dimensional contour and size of the locked product can be accurately and quickly measured, the target three-dimensional point cloud data can be obtained, and the locking state detection is performed on the target three-dimensional point cloud data, improving the accuracy of the locking state detection.

[0106] Based on the first embodiment of this application, in the second embodiment of this application, the same or similar content as in the above-mentioned first embodiment can be referred to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 5 , step S20 includes steps S301 to S303:

[0107] Step S301: Calculate the spatial normal vector of each point in the target three-dimensional point cloud data to obtain a set of normal vectors;

[0108] It can be understood that the normal vector is a unit vector perpendicular to the local surface where the point is located, and the direction usually points to the outside of the surface. In the three-dimensional point cloud data, the spatial normal vector is used to describe the geometric characteristics of the surface where each point in the point cloud is located. The spatial normal vector corresponding to each point can be calculated after searching for multiple nearest points for each point in the target three-dimensional point cloud data based on the KD tree search method to obtain a set of normal vectors.

[0109] In a feasible manner, the step of calculating the spatial normal vector of each point in the target three-dimensional point cloud data to obtain a set of normal vectors includes:

[0110] Obtain the set of neighboring points of each point in the target three-dimensional point cloud data;

[0111] Calculate the local covariance matrix through the set of neighboring points and perform eigenvalue decomposition to obtain eigenvectors;

[0112] Obtain the set of normal vectors according to the eigenvectors.

[0113] It is understandable that for each point in the searched target three-dimensional point cloud data, the number of its neighboring points is a preset value. After obtaining the set of neighboring points, the local covariance matrix is calculated and eigen-decomposed. The eigen-decomposition can obtain three eigenvalues and the corresponding three eigenvectors. Among them, the eigenvector corresponding to the smallest eigenvalue is the normal vector direction of the point. After obtaining the normal vector direction, we also need to normalize it. Normalization is to adjust the length of the normal vector to 1. Finally, the obtained normalized normal vectors are collected to form the normal vector set.

[0114] Step S302: Obtain a preset normal vector angle threshold, and cluster the target three-dimensional point cloud data by using the normal vector set and the preset normal vector angle threshold to obtain a point cloud clustering data set;

[0115] It is understandable that each point in the target three-dimensional point cloud data is clustered based on the similarity threshold between the corresponding normal vector angle and the preset normal vector angle threshold, so that the calculated normal vector angle difference between any two points in the cluster is not greater than the preset normal vector angle threshold, and a point cloud clustering data set is obtained.

[0116] Step S303: Screen out the initial point cloud of the reference plane and the initial point cloud of screw locking from the point cloud clustering data set according to a preset point cloud size.

[0117] It is understandable that after obtaining the point cloud clustering data set, the point cloud clustering data set is screened according to the preset point cloud size, and the point cloud clustering data that most conforms to the preset point cloud size is selected. Among them, the point cloud size can be judged by the number of points in the point cloud. The more the number of points, the larger the point cloud. The point cloud clustering data that most conforms to the preset point cloud size is the initial point cloud of the reference plane; and the remaining point cloud clustering data in the point cloud clustering data set is the initial point cloud of screw locking.

[0118] In this embodiment, the spatial normal vectors of each point in the target three-dimensional point cloud data are calculated to obtain a normal vector set; a preset normal vector angle threshold is obtained, and the target three-dimensional point cloud data is clustered by using the normal vector set and the preset normal vector angle threshold to obtain a point cloud clustering data set; the initial point cloud of the reference plane and the initial point cloud of screw locking are screened out from the point cloud clustering data set according to the preset point cloud size.

[0119] In summary, in this embodiment, the target three-dimensional point cloud data is classified by using the spatial normal vector of the target three-dimensional point cloud data and the preset normal vector threshold, and the initial point cloud of the reference plane can be accurately screened out, and the remaining point cloud is the initial point cloud of screw locking, which improves the accuracy of the locking state detection.

[0120] Exemplarily, to facilitate understanding of the implementation process of the attachment state detection method based on a laser profiler obtained by combining the present embodiment with the above-mentioned first embodiment, please refer to Figure 6 , Figure 6 A schematic diagram of the brief process of a method for detecting the attachment state based on a laser profiler is provided. Specifically: after starting the detection, first, point cloud data is obtained. Then, pass-through filtering and statistical filtering are performed on the point cloud data. After that, the normal vector of the point cloud data is calculated, and the reference plane and the measurement plane are screened and fitted. Next, the number of screws is counted, and the distance and angle of the screw plane are calculated to complete the detection of the attachment state. Finally, the result is output.

[0121] It should be noted that the above example is only for understanding the present application and does not constitute a limitation on the method for detecting the attachment state based on a laser profiler in the present application. Any simple transformation in more forms based on this technical concept is within the protection scope of the present application.

[0122] The present application also provides a device for detecting the attachment state based on a laser profiler. Please refer to Figure 7 , and the device for detecting the attachment state based on a laser profiler includes:

[0123] A scanning module 10, configured to scan an attached product through a laser profiler to obtain target three-dimensional point cloud data;

[0124] A classification module 20, configured to cluster and screen the target three-dimensional point cloud data to obtain initial point cloud of the reference plane and initial point cloud of screw attachment;

[0125] A fitting module 30, configured to perform plane fitting on the initial point cloud of the reference plane to obtain a reference plane;

[0126] A screening module 40, configured to cluster, screen, and fit the initial point cloud of screw attachment to obtain a set of measurement planes;

[0127] A determination module 50, configured to determine the attachment state of the attached product through the set of measurement planes and the reference plane.

[0128] The present embodiment provides a method for detecting the attachment state based on a laser profiler. The present application scans an attached product through a laser profiler to obtain target three-dimensional point cloud data; clusters and screens the target three-dimensional point cloud data to obtain initial point cloud of the reference plane and initial point cloud of screw attachment; performs plane fitting on the initial point cloud of the reference plane to obtain a reference plane; clusters, screens, and fits the initial point cloud of screw attachment to obtain a set of measurement planes; and determines the attachment state of the attached product through the set of measurement planes and the reference plane.

[0129] In summary, in this embodiment, by using a laser profiler to scan the locked product, the three-dimensional contour and dimensions of the locked product can be accurately and quickly measured, and the target three-dimensional point cloud data can be obtained, and the locked state is detected based on the target three-dimensional point cloud data, improving the accuracy of the locked state detection.

[0130] In one embodiment, the scanning module 10 is further configured to scan the locked product by using a laser profiler to obtain the original three-dimensional point cloud data of the locked product; obtain a preset three-dimensional coordinate range, and filter the original three-dimensional point cloud data through the preset three-dimensional coordinate range to obtain intermediate three-dimensional point cloud data; obtain the average distance between each point in the intermediate three-dimensional point cloud data and its surrounding neighboring points, and calculate the mean and standard deviation of the average distance; screen the intermediate three-dimensional point cloud data through the mean and standard deviation to obtain the target three-dimensional point cloud data.

[0131] In one embodiment, the classification module 20 is further configured to calculate the spatial normal vector of each point in the target three-dimensional point cloud data to obtain a normal vector set; obtain a preset normal vector angle threshold, and cluster the target three-dimensional point cloud data through the normal vector set and the preset normal vector angle threshold to obtain a point cloud clustering data set; screen out the initial point cloud of the reference plane and the initial point cloud of the screw lock from the point cloud clustering data set according to a preset point cloud size.

[0132] In one embodiment, the fitting module 30 is further configured to obtain the set of neighboring points of each point in the target three-dimensional point cloud data; calculate the local covariance matrix through the set of neighboring points and perform eigenvalue decomposition to obtain eigenvectors; obtain a normal vector set according to the eigenvectors.

[0133] In one embodiment, the fitting module 30 is further configured to calculate the center point coordinates of the initial point cloud of the reference plane; subtract the coordinates of each point in the initial point cloud of the reference plane from the center point coordinates to obtain target coordinates; determine the plane direction through the target coordinates;

[0134] Obtain the reference plane according to the center point coordinates and the plane direction.

[0135] In one embodiment, the screening module 40 is further configured to calculate the spatial normal vector of each point in the initial point cloud of the screw lock to obtain a screw point cloud normal vector set; obtain a preset screw point cloud normal vector angle threshold and a preset distance threshold; cluster according to the screw point cloud normal vector set and the preset screw point cloud angle, and the distance between any two points in the initial point cloud of the screw lock and the preset distance threshold to obtain a screw point cloud clustering data set; screen the screw point cloud clustering data set according to a preset screw point cloud size and a preset screw point cloud shape, and fit the screening result to obtain a set of measurement planes.

[0136] In one embodiment, the determination module 50 is further configured to obtain the number of measurement planes according to the set of measurement planes; obtain the angle and distance between each measurement plane and the reference plane according to the set of measurement planes and the reference plane; and determine the locking state of the locking product according to the number of measurement planes, the angle, and the distance.

[0137] The locking state detection device based on a laser profiler provided in this application adopts the locking state detection method based on a laser profiler in the above embodiment, and can solve the technical problem of how to improve the accuracy of locking state detection. Compared with the prior art, the beneficial effects of the locking state detection device based on a laser profiler provided in this application are the same as those of the locking state detection method based on a laser profiler provided in the above embodiment, and other technical features in the locking state detection device based on a laser profiler are the same as the features disclosed in the above embodiment method, and will not be elaborated here.

[0138] This application provides a locking state detection device based on a laser profiler. The locking state detection device based on a laser profiler includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the locking state detection method based on a laser profiler in the first embodiment above.

[0139] Reference is made below to Figure 8 , which shows a schematic structural diagram of a locking state detection device based on a laser profiler suitable for implementing the embodiments of this application. The locking state detection device based on a laser profiler in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The shown locking state detection device based on a laser profiler is only an example and should not impose any limitation on the functions and usage scope of the embodiments of this application.

[0140] As Figure 8As shown, the attachment state detection device based on a laser profiler may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in a ROM (Read Only Memory) 1002 or a program loaded from a storage device 1003 into a RAM (Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the attachment state detection device based on a laser profiler are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the attachment state detection device based on a laser profiler to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows an attachment state detection device based on a laser profiler having various systems, it should be understood that it is not required to implement or have all the shown systems. More or fewer systems may be alternatively implemented or had.

[0141] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments disclosed in the present application are executed.

[0142] The attachment state detection device based on a laser profiler provided by this application adopts the attachment state detection method based on a laser profiler in the above-mentioned embodiment, and can solve the technical problem of how to improve the accuracy of attachment state detection. Compared with the prior art, the beneficial effects of the attachment state detection device based on a laser profiler provided by this application are the same as those of the attachment state detection method based on a laser profiler provided by the above-mentioned embodiment, and other technical features in the attachment state detection device based on a laser profiler are the same as the features disclosed in the method of the previous embodiment, and will not be elaborated here.

[0143] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0144] As mentioned above, the above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application, and all of them should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0145] This application provides a computer-readable storage medium with computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the attachment state detection method based on a laser profiler in the above-mentioned embodiment.

[0146] The computer-readable storage medium provided by the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0147] The above computer-readable storage medium may be included in a lock state detection device based on a laser profiler; or may exist alone without being assembled into a lock state detection device based on a laser profiler.

[0148] The above computer-readable storage medium carries one or more programs. When the one or more programs are executed by a lock state detection device based on a laser profiler, the lock state detection device based on a laser profiler is caused to: scan a locked product through a laser profiler to obtain target three-dimensional point cloud data; perform clustering and screening on the target three-dimensional point cloud data to obtain an initial point cloud of a reference plane and an initial point cloud of screw locking; perform plane fitting on the initial point cloud of the reference plane to obtain a reference plane; perform clustering, screening, and fitting on the initial point cloud of screw locking to obtain a set of measurement planes; and determine the locking state of the locked product through the set of measurement planes and the reference plane.

[0149] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The above-mentioned programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0150] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0151] The modules involved in the embodiments described in this application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation to the unit itself in some cases.

[0152] The readable storage medium provided in this application is a computer-readable storage medium. The computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for performing the above-mentioned locking state detection method based on a laser profiler, and can solve the technical problem of how to improve the accuracy of locking state detection. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the locking state detection method based on a laser profiler provided in the above embodiments, and will not be elaborated here.

[0153] The present application also provides a computer program product, including a computer program which, when executed by a processor, implements the steps of the above-described method for detecting the locking state based on a laser profiler.

[0154] The computer program product provided by the present application can solve the technical problem of how to improve the accuracy of detecting the locking state. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as those of the method for detecting the locking state based on a laser profiler provided in the above embodiments, and will not be elaborated herein.

[0155] The above are only partial embodiments of the present application, and thus do not limit the patent scope of the present application. Any equivalent structural transformation made by using the content of the specification and drawings of the present application under the technical concept of the present application, or any direct / indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. A method for detecting the locking state based on a laser profiler, characterized in that, The described method includes: Scanning the locked product with a laser profiler to obtain target three-dimensional point cloud data; Clustering and screening the target three-dimensional point cloud data to obtain the initial point cloud of the reference plane and the initial point cloud of screw locking; Performing plane fitting on the initial point cloud of the reference plane to obtain the reference plane; Clustering, screening, and fitting the initial point cloud of screw locking to obtain a set of measurement planes; Determining the locking state of the locked product through the set of measurement planes and the reference plane.

2. The method according to claim 1, characterized in that, The step of scanning the locked product with a laser profiler to obtain target three-dimensional point cloud data includes: Scanning the locked product with a laser profiler to obtain the original three-dimensional point cloud data of the locked product; Obtaining a preset three-dimensional coordinate range and filtering the original three-dimensional point cloud data through the preset three-dimensional coordinate range to obtain intermediate three-dimensional point cloud data; Obtaining the average distance between each point in the intermediate three-dimensional point cloud data and its surrounding neighboring points, and calculating the mean and standard deviation of the average distance; Screening the intermediate three-dimensional point cloud data through the mean and standard deviation to obtain the target three-dimensional point cloud data.

3. The method according to claim 1, characterized in that, The step of clustering and screening the target three-dimensional point cloud data to obtain the initial point cloud of the reference plane and the initial point cloud of screw locking includes: Calculating the spatial normal vector of each point in the target three-dimensional point cloud data to obtain a set of normal vectors; Obtaining a preset normal vector angle threshold, and clustering the target three-dimensional point cloud data through the set of normal vectors and the preset normal vector angle threshold to obtain a set of point cloud clustering data; Selecting the initial point cloud of the reference plane and the initial point cloud of screw locking from the set of point cloud clustering data according to a preset point cloud size.

4. The method according to claim 3, characterized in that, The step of calculating the spatial normal vector of each point in the target three-dimensional point cloud data to obtain a set of normal vectors includes: Obtaining the set of neighboring points of each point in the target three-dimensional point cloud data; Calculating the local covariance matrix through the set of neighboring points and performing eigenvalue decomposition to obtain the eigenvectors; Obtaining the set of normal vectors according to the eigenvectors.

5. The method according to claim 1, characterized in that, The step of performing plane fitting on the initial point cloud of the reference plane to obtain the reference plane includes: Calculating the center point coordinates of the initial point cloud of the reference plane; Subtracting the center point coordinates from the coordinates of each point in the initial point cloud of the reference plane to obtain the target coordinates; Determining the plane direction through the target coordinates; Obtaining the reference plane according to the center point coordinates and the plane direction.

6. The method according to claim 1, wherein The step of clustering, screening, and fitting the initial point cloud of screw locking to obtain a set of measurement planes includes: Calculating the spatial normal vector of each point in the initial point cloud of screw locking to obtain a set of screw point cloud normal vectors; Obtaining a preset screw point cloud normal vector angle threshold and a preset distance threshold; Clustering according to the set of screw point cloud normal vectors and the preset screw point cloud normal vector angle, and the distance between any two points in the initial point cloud of screw locking and the preset distance threshold to obtain a set of screw point cloud clustering data; Screening the set of screw point cloud clustering data according to a preset screw point cloud size and a preset screw point cloud shape, and fitting the screening result to obtain a set of measurement planes.

7. The method according to claim 1, characterized in that The step of determining the locking state of the locking product by the set of measurement planes and the reference plane includes: Obtaining the number of measurement planes according to the set of measurement planes; Obtaining the angle and distance between each measurement plane and the reference plane according to the set of measurement planes and the reference plane; Determining the locking state of the locking product according to the number of measurement planes, the angle and the distance.

8. A locking state detection device based on a laser profiler, characterized in that, The device includes: A scanning module, configured to scan the locking product through a laser profiler to obtain target three-dimensional point cloud data; A classification module, configured to cluster and screen the target three-dimensional point cloud data to obtain an initial point cloud of the reference plane and an initial point cloud of screw locking; A fitting module, configured to perform plane fitting on the initial point cloud of the reference plane to obtain a reference plane; A screening module, configured to cluster, screen and fit the initial point cloud of screw locking to obtain a set of measurement planes; A determining module, configured to determine the locking state of the locking product by the set of measurement planes and the reference plane.

9. A locking state detection device based on a laser profiler, characterized in that The device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the computer program is configured to implement the steps of the locking state detection method based on a laser profiler according to any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the locking state detection method based on a laser profiler according to any one of claims 1 to 7.

Images