Nut loosening degree detection method, medium and system based on 3D point cloud data

Abstract

The application discloses a nut loosening degree detection method, medium and system based on 3D point cloud data, and the method comprises the following steps: S1, a vehicle with a fault-free nut is selected to obtain corresponding picture point cloud data as template point cloud data, and picture point cloud data of a nut to be detected is obtained as detected point cloud data; S2, a corresponding detected ROI region in the detected point cloud data is cropped according to a template ROI region determined in the template point cloud data; S3, the template ROI region and the detected ROI region are registered; S4, the position of the nut in the detected ROI region point cloud is obtained by processing the detected ROI region point cloud according to the template ROI region point cloud; and S5, the distance between the position point cloud of the nut and the base plane is obtained, and then the loosening degree of the nut is judged according to the distance. The application can quantify the loosening degree and has high detection precision.

Description

Technical Field

[0001] The present invention mainly relates to the technical field of nut looseness detection, and specifically to a nut looseness degree detection method, medium and system based on 3D point cloud data. Background Art

[0002] Nuts, as a common fastener, are widely used in vehicles such as locomotives, urban rail transit, and EMUs. The stability of nut tightening affects train safety, making looseness detection crucial. Currently, locomotive depots and railway bureaus still rely on manual inspection, using auxiliary tools such as anti-loosening markings. Thousands of nuts are tightened on a single train, and repeated inspections require significant time and manpower, making them prone to oversight.

[0003] Currently, methods for identifying loose nuts are primarily divided into sensor-based methods and 2D image data-based methods. Sensor-based methods typically require the installation of sensors on the components to be inspected, which can be susceptible to environmental limitations. 2D image data-based methods, on the other hand, are susceptible to factors such as water stains, dust, dirt, and lighting on the vehicle surface. Summary of the Invention

[0004] The technical problem to be solved by the present invention is: in response to the technical problems existing in the prior art, the present invention provides a method, medium and system for detecting the looseness of a nut based on 3D point cloud data with high detection accuracy.

[0005] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0006] A method for detecting the looseness of a nut based on 3D point cloud data, comprising the steps of:

[0007] S1. Select a vehicle with no faulty nuts to obtain the corresponding image point cloud data as template point cloud data, and obtain the image point cloud data of the nut to be inspected as the point cloud data to be inspected;

[0008] S2. Cut out the corresponding ROI area to be detected in the point cloud data to be inspected according to the template ROI area determined in the template point cloud data;

[0009] S3, registering the template ROI area with the ROI area to be detected;

[0010] S4. Obtain the position of the nut in the point cloud of the ROI area to be detected according to the point cloud of the template ROI area;

[0011] S5. Obtain the distance between the point cloud of the nut location and the base plane, and then determine the degree of looseness of the nut based on the distance.

[0012] Preferably, the specific process of step S4 is:

[0013] Perform plane fitting on the point cloud of the ROI area to be inspected, and distinguish the base plane point cloud where the nut is installed from other point clouds;

[0014] Cluster other point clouds to obtain point cloud clusters with category labels;

[0015] Based on the position information of each nut in the template ROI area point cloud, the area is expanded in the x, y, and z directions respectively; based on the expanded space, the spatial point cloud that may contain the nut to be detected is intercepted from the point cloud of the ROI area to be detected;

[0016] The point cloud cluster is intersected with the spatial point cloud, and the obtained intersection constitutes the position of the nut to be detected.

[0017] Preferably, the specific process of step S5 is:

[0018] Calculate the distance from the point cloud at the nut location to the base plane, sort the distance set from high to low, and remove points with a distance less than H2 to obtain the distance array dist_h:[h1,h2,h3,h4,.....,hn];

[0019] If all values ​​in dist_h are less than H1+0.5, then remove the values ​​in dist_h that are greater than H1-0.5. Find the highest plane parallel to the base among the remaining points, record its plane height H0, and calculate H0-H2 to be the degree of nut looseness.

[0020] If there are more than 10 points in dist_h that are greater than H1+0.5, find the highest plane parallel to the base among all the points in dist_h, record its plane height H00, and calculate H00-H2 to get the degree of nut looseness;

[0021] Wherein H1 is the vertical distance between the first plane where the top of the bolt is located and the bottom plane in the normal state; H2 is the vertical distance between the second plane where the top of the nut is located and the bottom plane in the normal state.

[0022] Preferably, in the process of searching for a plane, points with a height difference from the base plane less than 0.1 are considered to be in the same plane.

[0023] Preferably, H1 and H2 are calculated for each nut separately based on the template image point cloud data.

[0024] Preferably, in step S3, before registration, the template ROI region and the ROI region to be detected are down-sampled.

[0025] Preferably, the ICP registration method is used for registration.

[0026] Preferably, in step S1 , the corresponding template point cloud data and the point cloud data to be inspected are of the same vehicle type, the same shooting angle and the same position.

[0027] The present invention also discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described above are executed.

[0028] The present invention further discloses a nut looseness detection system based on 3D point cloud data, comprising a memory and a processor connected to each other, wherein a computer program is stored on the memory, and when the computer program is run by the processor, the steps of the method described above are executed.

[0029] Compared with the prior art, the advantages of the present invention are:

[0030] The present invention uses only 3D images / point clouds for detection of loose nuts on trains, reducing system complexity and hardware costs while ensuring the detection rate of loose nuts. Since loose nuts are identified through spatial differences based on three-dimensional point cloud data, they are less affected by light and body stains, and have high detection accuracy.

[0031] The present invention adopts a template matching method to locate the nut to be inspected, analyzes the difference between the point cloud data to be inspected and the template point cloud data, thereby identifying looseness and reducing the workload of manual annotation. It is more applicable to items with a small demand for negative samples or a lack of negative samples. The image to be inspected and the local area of ​​the template image are aligned, the base plane is removed, the other part of the point cloud is clustered, and the target position is accurately located in combination with the original template detection. In the event of registration failure, the positioning accuracy of the target to be inspected is improved, thereby enhancing the fault tolerance of the model. The above method quantifies the degree of looseness based on the difference in 3D point cloud data between the target to be inspected and the template target, and intuitively gives the looseness value. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 The figure is a flow chart of the detection method of the present invention in an embodiment.

[0033] Figure 2 It is a schematic diagram of the base plane in an embodiment of the present invention.

[0034] Figure 3 It is a schematic diagram of the first plane, the second plane and the bottom plane in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0036] like Figure 1As shown, the nut loosening degree detection method based on 3D point cloud data of the embodiment of the present application is suitable for repetitive detection of specific objects, and specifically includes the following steps:

[0037] S1, data acquisition: first, according to the vehicle type, the corresponding template is made, the point cloud data of the vehicle without nut failure is selected as the template point cloud data, and the picture point cloud data of the nut to be detected is obtained as the to-be-detected point cloud data; wherein the to-be-detected point cloud data and the template point cloud data are point cloud data obtained at different times for the same vehicle type, the same shooting angle, the same position. Due to the shaking of the collection device, the train shaking or the installation error, there is a certain deviation between the corresponding template point cloud data and the to-be-detected point cloud data.

[0038] S2, ROI region cropping: for each template point cloud data, the position information of the template ROI region and the nut is determined. In the corresponding to-be-detected point cloud data, the corresponding to-be-detected ROI region (target ROI region) is cropped according to the ROI region determined in the template point cloud data.

[0039] S3, registration: in order to improve the calculation efficiency, the template ROI region and the target ROI region are first down-sampled, and then registered to make the coincidence degree as high as possible to remove the nut installation base plane point cloud; wherein the ICP registration method is adopted in this example;

[0040] S4, to-be-detected target positioning: the target ROI region point cloud is fitted (such as using RANSAC), and the nut installation base plane point cloud and other point clouds are further distinguished;

[0041] The other point clouds are clustered (such as using the dbscan clustering algorithm), and the point cloud cluster 1 with a category label is obtained;

[0042] According to the position information (the lower left point and the upper right point of the point cloud solid frame) of each nut in the template point cloud data, the region is appropriately expanded in x, y and z directions; correspondingly, according to the expansion space, the space point cloud 2 containing the to-be-detected nut is intercepted from the point cloud in the target ROI region;

[0043] The expanded space point cloud 2 and the point cloud cluster 1 are intersected, and the intersection points constitute the fine position of the to-be-detected nut.

[0044] S5, loosening identification: for each nut in the template point cloud data, the distance from the first plane and the second plane to the base plane is calculated respectively, and is recorded and saved as H1 and H2; such as Figure 3 As shown, wherein H1 is the vertical distance between the first plane where the bolt top is located and the bottom plane in the normal state; H1 is the vertical distance between the second plane where the nut top is located and the bottom plane in the normal state;

[0045] Based on the distances obtained in the previous step and the nut point cloud without the base plane, calculate the distances from all points to the base plane, sort the distance set from high to low, and remove points with distances less than H2 to obtain the distance array dist_h:[h1,h2,h3,h4,.....,hn]; search for a plane parallel to the base plane in the remaining point cloud;

[0046] If all values ​​in dist_h are less than H1+0.5mm (this value can be determined based on the specific conditions such as the nut type), it indicates that the top end surface of the nut (the second plane) does not extend beyond the end plane of the bolt (the first plane). In this case, remove the values ​​in dist_h that are greater than H1-0.5. Find the highest plane parallel to the base among the remaining points, record its plane height H0, and calculate H0-H2 to determine the degree of nut looseness.

[0047] If there are more than 10 points in dist_h that are larger than H1+0.5mm, it indicates that the top end face of the nut (the second plane) has exceeded the end plane of the bolt (the first plane). Find the highest plane parallel to the base among all points in dist_h, record its plane height H00, and calculate H00-H2 to be the degree of nut looseness.

[0048] When searching for the highest plane parallel to the base, points with a height difference from the base plane less than 0.1 are considered to be on the same plane.

[0049] The present invention uses only 3D images / point clouds for detection of loose nuts on trains, reducing system complexity and hardware costs while ensuring the detection rate of loose nuts. Since loose nuts are identified through spatial differences based on three-dimensional point cloud data, they are less affected by light and body stains, and have high detection accuracy.

[0050] The present invention adopts a template matching method to locate the nut to be inspected, analyzes the difference between the point cloud data to be inspected and the template point cloud data, thereby identifying looseness and reducing the workload of manual annotation. It is more applicable to items with a small demand for negative samples or a lack of negative samples. The image to be inspected and the local area of ​​the template image are aligned, the base plane is removed, the other part of the point cloud is clustered, and the target position is accurately located in combination with the original template detection. In the event of registration failure, the positioning accuracy of the target to be inspected is improved, thereby enhancing the fault tolerance of the model. The above method quantifies the degree of looseness based on the difference in 3D point cloud data between the target to be inspected and the template target, and intuitively gives the looseness value.

[0051] An embodiment of the present invention also discloses a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program performs the steps of the above-described method. An embodiment of the present invention further discloses a nut looseness detection system based on 3D point cloud data, comprising an interconnected memory and a processor. The memory has a computer program stored thereon. When executed by the processor, the computer program performs the steps of the above-described method. The medium and system of the present invention correspond to the above-described method and similarly possess the advantages described by the above-described method.

[0052] The present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned method embodiment. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. Computer-readable media include: any entity or device that can carry computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. The memory is used to store computer programs and / or modules, and the processor implements various functions by running or executing computer programs and / or modules stored in the memory, and calling data stored in the memory. The memory may include high-speed random access memory and non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0053] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions based on the principles of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for detecting the degree of nut looseness based on 3D point cloud data, characterized in that: Including steps: S1. Select a vehicle with no faulty nuts to obtain the corresponding image point cloud data as template point cloud data, and obtain the image point cloud data of the nut to be inspected as the point cloud data to be inspected; S2. Cut out the corresponding ROI area to be detected in the point cloud data to be inspected according to the template ROI area determined in the template point cloud data; S3, registering the template ROI area with the ROI area to be detected; S4. Processing the point cloud of the ROI region to be detected based on the point cloud of the template ROI region to obtain the position of the nut in the point cloud of the ROI region to be detected; S5. Obtain the distance between the point cloud of the nut location and the base plane, and then determine the degree of looseness of the nut based on the distance; The specific process of step S5 is: Calculate the distance from the point cloud at the nut location to the base plane, sort the distance set from high to low, and remove points with a distance less than H2 to obtain the distance array dist_h:[h1,h2,h3,h4,.....,hn]; If all values ​​in dist_h are less than H1+0.5, then remove the values ​​in dist_h that are greater than H1-0.

5. Find the highest plane parallel to the base among the remaining points, record its plane height H0, and calculate H0-H2 to be the degree of nut looseness. If there are more than 10 points in dist_h that are greater than H1+0.5, find the highest plane parallel to the base among all the points in dist_h, record its plane height H00, and calculate H00-H2 to get the degree of nut looseness; Wherein H1 is the vertical distance between the first plane where the top of the bolt is located and the bottom plane in the normal state; H2 is the vertical distance between the second plane where the top of the nut is located and the bottom plane in the normal state.

2. The method for detecting the degree of nut looseness based on 3D point cloud data according to claim 1, characterized in that: The specific process of step S4 is: Perform plane fitting on the point cloud of the ROI area to be inspected, and distinguish the base plane point cloud where the nut is installed from other point clouds; Cluster other point clouds to obtain point cloud clusters with category labels; Based on the position information of each nut in the template ROI area point cloud, the area is expanded in the x, y, and z directions respectively; based on the expanded space, the spatial point cloud that may contain the nut to be detected is intercepted from the point cloud of the ROI area to be detected; The point cloud cluster is intersected with the spatial point cloud, and the obtained intersection constitutes the position of the nut to be detected.

3. The method for detecting the degree of nut looseness based on 3D point cloud data according to claim 1, characterized in that: In the process of searching for a plane, points with a height difference from the base plane less than 0.1 are considered to be on the same plane.

4. The method for detecting the degree of nut looseness based on 3D point cloud data according to claim 1, characterized in that: Among them, H1 and H2 are calculated for each nut based on the template image point cloud data.

5. The method for detecting nut looseness based on 3D point cloud data according to any one of claims 1 to 4, characterized in that: In step S3, before registration, the template ROI region and the ROI region to be detected are down-sampled.

6. The method for detecting the degree of nut looseness based on 3D point cloud data according to claim 5, characterized in that: The ICP registration method is used for registration.

7. The method for detecting nut looseness based on 3D point cloud data according to any one of claims 1 to 4, characterized in that: In step S1 , the corresponding template point cloud data and the point cloud data to be inspected are of the same vehicle type, the same shooting angle and the same position.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program performs the steps of the method according to any one of claims 1 to 7.

9. A nut looseness detection system based on 3D point cloud data, comprising a memory and a processor connected to each other, wherein a computer program is stored in the memory, characterized in that: When the computer program is executed by a processor, the computer program performs the steps of the method according to any one of claims 1 to 7.

Images