Nut loosening degree detection method based on 3D point cloud data, medium and system

Abstract

The invention discloses a nut looseness degree detection method based on 3D point cloud data, a medium and a system, and the method comprises the steps: S1, selecting a vehicle without a fault nut, obtaining the corresponding image point cloud data as template point cloud data, and obtaining the image point cloud data of a to-be-detected nut as to-be-detected point cloud data; s2, cutting out a corresponding ROI region to be detected in the point cloud data to be detected according to a template ROI region determined in the template point cloud data; s3, carrying out registration on the template ROI region and the ROI region to be detected; s4, according to the template ROI point cloud, processing the to-be-detected ROI point cloud to obtain the position of a nut in the to-be-detected ROI point cloud; s5, the distance between the point cloud of the position where the nut is located and the plane of the base is obtained, and the looseness degree of the nut is judged according to the distance. The looseness degree can be quantified, and the detection precision is high.

Description

Technical Field

[0001] The present invention mainly relates to the technical field of nut looseness detection, and specifically relates to a nut looseness degree detection method, medium and system based on 3D point cloud data. Background Art

[0002] As a common fastener, nuts are widely used in locomotives, urban rails, EMUs and other vehicles. The stability of nut tightening is related to the safety of train operation, so it is very important to detect whether the nuts are tightened securely. At present, each locomotive section and railway bureau still uses manual detection and identification through auxiliary means such as anti-loosening marking lines. There are tens of thousands of nuts tightened on a train, and repeated inspections of the train require a lot of time and manpower, which is prone to human negligence.

[0003] At present, the identification methods of nut loosening are mainly divided into sensor-based methods and two-dimensional image data-based methods. Sensor-based methods usually require the installation of sensor devices on the components to be detected, which are easily restricted by the environment; while two-dimensional image data-based methods are easily affected by factors such as water stains, dust, stains, and light on the surface of the vehicle body. Summary of the invention

[0004] The technical problem to be solved by the present invention is: in view of the technical problems existing in the prior art, the present invention provides a method, medium and system for detecting the looseness of a nut based on 3D point cloud data with high detection accuracy.

[0005] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0006] A method for detecting the looseness of a nut based on 3D point cloud data, comprising the steps of:

[0007] S1. Select a vehicle with no faulty nuts to obtain corresponding image point cloud data as template point cloud data, and obtain image point cloud data of nuts to be inspected as point cloud data to be inspected;

[0008] S2, cutting out the corresponding ROI area to be detected in the point cloud data to be inspected according to the template ROI area determined in the template point cloud data;

[0009] S3, registering the template ROI area with the ROI area to be detected;

[0010] S4, obtaining the position of the nut in the point cloud of the ROI area to be detected according to the point cloud of the template ROI area;

[0011] S5. Obtain the distance between the point cloud of the nut location and the base plane, and then determine the degree of looseness of the nut based on the distance.

[0012] Preferably, the specific process of step S4 is:

[0013] Perform plane fitting on the point cloud of the ROI area to be inspected, and distinguish the base plane point cloud where the nut is installed from other point clouds;

[0014] Cluster other point clouds to obtain point cloud clusters with category labels;

[0015] According to the position information of each nut in the point cloud of the template ROI area, the area is expanded in the three directions of x, y, and z respectively; according to the expanded space, the spatial point cloud that may contain the nut to be detected is intercepted in the point cloud of the ROI area to be detected;

[0016] The point cloud cluster is intersected with the spatial point cloud, and the obtained intersection points constitute the positions of the nuts to be detected.

[0017] Preferably, the specific process of step S5 is:

[0018] Calculate the distance from the point cloud where the nut is located to the base plane, sort the distance set from high to low, and remove the points with a distance less than H2 to obtain the distance array dist_h: [h1, h2, h3, h4, ....., hn];

[0019] If all values ​​in dist_h are less than H1+0.5, then remove the values ​​in dist_h that are greater than H1-0.5, find the highest plane parallel to the base among the remaining points, record its plane height H0, and calculate H0-H2 to be the degree of nut looseness;

[0020] If there are more than 10 points in dist_h that are greater than H1+0.5, find the plane that is parallel to the base and has the highest height among all the points in dist_h, record its plane height H00, and calculate H00-H2 to be the degree of nut looseness;

[0021] Wherein H1 is the vertical distance between the first plane where the top of the bolt is located and the bottom plane in the normal state; H2 is the vertical distance between the second plane where the top of the nut is located and the bottom plane in the normal state.

[0022] Preferably, in the process of searching for a plane, points with a height difference from the base plane less than 0.1 are considered to be in the same plane.

[0023] Preferably, H1 and H2 are calculated for each nut separately according to the template image point cloud data.

[0024] Preferably, in step S3, before registration, the template ROI region and the ROI region to be detected are down-sampled.

[0025] Preferably, the ICP registration method is used for registration.

[0026] Preferably, in step S1, the corresponding template point cloud data and the point cloud data to be inspected are of the same vehicle type, the same shooting angle and the same position.

[0027] The present invention also discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described above are executed.

[0028] The present invention further discloses a nut looseness detection system based on 3D point cloud data, comprising a memory and a processor connected to each other, wherein a computer program is stored on the memory, and when the computer program is run by the processor, the steps of the method described above are executed.

[0029] Compared with the prior art, the advantages of the present invention are:

[0030] The present invention only uses 3D images / point clouds to detect the loose nut failure of trains, which reduces the system complexity and hardware cost while ensuring the detection rate of loose nut. Since it is based on three-dimensional point cloud data and identifies loose nuts through spatial differences, it is less affected by light and body stains and has high detection accuracy.

[0031] The present invention adopts a template matching method to locate the nut to be detected, analyzes the difference between the point cloud data to be detected and the template point cloud data, thereby identifying looseness, reducing the workload of manual annotation, and having better applicability for items with less negative sample demand and lack of negative samples; aligns the local area of ​​the image to be detected with the template image, removes the base plane, clusters the other part of the point cloud, and accurately locates the target position in combination with the original template detection; in the case of registration failure, improves the positioning accuracy of the target to be detected, thereby enhancing the fault tolerance of the model. The above method quantifies the degree of looseness according to the difference in 3D point cloud data between the target to be detected and the template target, and intuitively gives the looseness value. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 The present invention is a flow chart of the detection method in an embodiment.

[0033] Figure 2 It is a schematic diagram of the base plane of the present invention in an embodiment.

[0034] Figure 3 It is a schematic diagram of the first plane, the second plane and the bottom plane in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0036] like Figure 1As shown, the nut looseness detection method based on 3D point cloud data of an embodiment of the present invention is suitable for repeatability detection of a specific object, and specifically includes the following steps:

[0037] S1. Data acquisition: First, make a corresponding template according to the vehicle model, select a vehicle without nut failure to obtain the corresponding point cloud data as the template point cloud data, and obtain the image point cloud data of the nut to be detected as the point cloud data to be inspected; the point cloud data to be inspected and the template point cloud data are point cloud data obtained at different times for the same vehicle model, the same shooting angle, the same position. Due to the jitter of the acquisition equipment, the shaking of the train or the installation error, there is a certain deviation between the corresponding template point cloud data and the point cloud data to be inspected.

[0038] S2. ROI area cropping: For each template point cloud data, determine the template ROI area and the position information of the nut. In the corresponding point cloud data to be inspected, the corresponding ROI area to be inspected (target ROI area) is cropped according to the ROI area determined in the template point cloud data.

[0039] S3, registration: In order to improve the calculation efficiency, the template ROI area and the target ROI area are first downsampled and then registered to make their overlap as high as possible to remove the base plane point cloud where the nut is installed; in this example, the ICP registration method is used;

[0040] S4. Positioning the target to be inspected: performing plane fitting (such as using RANSAC) on the point cloud of the target ROI area, and further distinguishing the base plane point cloud where the nut is installed from other point clouds;

[0041] Cluster other point clouds (such as using the dbscan clustering algorithm) to obtain point cloud cluster 1 with category labels;

[0042] According to the position information of each nut in the template point cloud data (the lower left point and the upper right point of the point cloud stereo frame), the area is appropriately expanded in the three directions of x, y, and z; correspondingly, according to the expanded space, the spatial point cloud 2 that may contain the nut to be detected is intercepted in the point cloud of the target ROI area;

[0043] The expanded spatial point cloud 2 is intersected with the point cloud cluster 1, and the obtained intersection constitutes the precise position of the nut to be detected.

[0044] S5, loose identification: for each nut in the template point cloud data, calculate the distances from the first plane and the second plane to the base plane respectively, and record and save them as H1 and H2; Figure 3 As shown, H1 is the vertical distance between the first plane where the top of the bolt is located and the bottom plane in the normal state; H2 is the vertical distance between the second plane where the top of the nut is located and the bottom plane in the normal state;

[0045] According to the distance obtained in the previous step and the nut point cloud without the base plane, calculate the distance from all points to the base plane, sort the distance set from high to low, remove the points with a distance less than H2, and obtain the distance array dist_h: [h1, h2, h3, h4, ....., hn]; find the plane parallel to the base plane in the remaining point cloud;

[0046] If all values ​​in dist_h are less than H1+0.5mm (this value can be determined according to the specific conditions such as the nut type), it means that the top end face of the nut (the second plane) does not exceed the end plane of the bolt (the first plane). Then, the values ​​in dist_h that are greater than H1-0.5 are eliminated, and the plane with the highest height parallel to the base is found among the remaining points. Its plane height H0 is recorded, and H0-H2 is calculated to be the degree of nut looseness.

[0047] If there are more than 10 points in dist_h that are larger than H1+0.5mm, it indicates that the top end face of the nut (the second plane) has exceeded the end plane of the bolt (the first plane). Find the plane that is parallel to the base and has the highest height among all points in dist_h, record its plane height H00, and calculate H00-H2 to be the degree of nut looseness.

[0048] When searching for the plane that is parallel to the base and has the highest height, points with a height difference from the base plane that is less than 0.1 are considered to be in the same plane.

[0049] The present invention only uses 3D images / point clouds to detect the loose nut failure of trains, which reduces the system complexity and hardware cost while ensuring the detection rate of loose nut. Since it is based on three-dimensional point cloud data and identifies loose nuts through spatial differences, it is less affected by light and body stains and has high detection accuracy.

[0050] The present invention adopts a template matching method to locate the nut to be detected, analyzes the difference between the point cloud data to be detected and the template point cloud data, thereby identifying looseness, reducing the workload of manual annotation, and having better applicability for items with less negative sample demand and lack of negative samples; aligns the local area of ​​the image to be detected with the template image, removes the base plane, clusters the other part of the point cloud, and accurately locates the target position in combination with the original template detection; in the case of registration failure, improves the positioning accuracy of the target to be detected, thereby enhancing the fault tolerance of the model. The above method quantifies the degree of looseness according to the difference in 3D point cloud data between the target to be detected and the template target, and intuitively gives the looseness value.

[0051] The embodiment of the present invention further discloses a computer-readable storage medium, on which a computer program is stored, and the computer program executes the steps of the above method when it is executed by a processor. The embodiment of the present invention further discloses a nut looseness detection system based on 3D point cloud data, including a memory and a processor connected to each other, and the memory stores a computer program, and the computer program executes the steps of the above method when it is executed by the processor. The medium and system of the present invention correspond to the above method and also have the advantages described in the above method.

[0052] The present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiment can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. Computer-readable media include: any entity or device that can carry computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. The memory is used to store computer programs and / or modules, and the processor implements various functions by running or executing computer programs and / or modules stored in the memory, and calling data stored in the memory. The memory may include high-speed random access memory and may also include non-volatile memory, such as a hard disk, an internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0053] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A method for detecting the looseness of a nut based on 3D point cloud data, characterized in that: Includes steps: S1. Select a vehicle with no faulty nuts to obtain corresponding image point cloud data as template point cloud data, and obtain image point cloud data of nuts to be inspected as point cloud data to be inspected; S2, cutting out the corresponding ROI area to be detected in the point cloud data to be inspected according to the template ROI area determined in the template point cloud data; S3, registering the template ROI area with the ROI area to be detected; S4, processing the point cloud of the ROI area to be detected according to the point cloud of the template ROI area to obtain the position of the nut in the point cloud of the ROI area to be detected; S5. Obtain the distance between the point cloud of the nut location and the base plane, and then determine the degree of looseness of the nut based on the distance.

2. The method for detecting the degree of nut looseness based on 3D point cloud data according to claim 1 is characterized in that: The specific process of step S4 is: Perform plane fitting on the point cloud of the ROI area to be inspected, and distinguish the base plane point cloud where the nut is installed from other point clouds; Cluster other point clouds to obtain point cloud clusters with category labels; According to the position information of each nut in the point cloud of the template ROI area, the area is expanded in the three directions of x, y, and z respectively; according to the expanded space, the spatial point cloud that may contain the nut to be detected is intercepted in the point cloud of the ROI area to be detected; The point cloud cluster is intersected with the spatial point cloud, and the obtained intersection points constitute the positions of the nuts to be detected.

3. The method for detecting the degree of nut looseness based on 3D point cloud data according to claim 1 is characterized in that: The specific process of step S5 is: Calculate the distance from the point cloud where the nut is located to the base plane, sort the distance set from high to low, and remove the points with a distance less than H2 to obtain the distance array dist_h: [h1, h2, h3, h4, ....., hn]; If all values ​​in dist_h are less than H1+0.5, then remove the values ​​in dist_h that are greater than H1-0.5, find the highest plane parallel to the base among the remaining points, record its plane height H0, and calculate H0-H2 to be the degree of nut looseness; If there are more than 10 points in dist_h that are greater than H1+0.5, find the plane that is parallel to the base and has the highest height among all the points in dist_h, record its plane height H00, and calculate H00-H2 to be the degree of nut looseness; Wherein H1 is the vertical distance between the first plane where the top of the bolt is located and the bottom plane in the normal state; H2 is the vertical distance between the second plane where the top of the nut is located and the bottom plane in the normal state.

4. The method for detecting the degree of nut looseness based on 3D point cloud data according to claim 3 is characterized in that: In the process of searching for a plane, points with a height difference from the base plane less than 0.1 are considered to be in the same plane.

5. The method for detecting the degree of nut looseness based on 3D point cloud data according to claim 3 is characterized in that: Among them, H1 and H2 are calculated for each nut according to the point cloud data of the template image.

6. The method for detecting the degree of nut looseness based on 3D point cloud data according to any one of claims 1 to 5, characterized in that: In step S3, before registration, the template ROI region and the ROI region to be detected are down-sampled.

7. The method for detecting the degree of nut looseness based on 3D point cloud data according to claim 6 is characterized in that: The ICP registration method was used for registration.

8. The method for detecting the degree of nut looseness based on 3D point cloud data according to any one of claims 1 to 5, characterized in that: In step S1 , the corresponding template point cloud data and the to-be-checked point cloud data are of the same vehicle type, the same shooting angle and the same position.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program executes the steps of the method according to any one of claims 1 to 8 when executed by a processor.

10. A nut looseness detection system based on 3D point cloud data, comprising a memory and a processor connected to each other, wherein a computer program is stored in the memory, characterized in that: The computer program executes the steps of the method according to any one of claims 1 to 8 when executed by a processor.

Images