Method, device and terminal for detecting looseness of hexagon bolt on railway vehicle

Abstract

The invention provides a method, a device and a terminal for detecting looseness of a hexagon bolt on a railway vehicle. The method comprises the following steps: acquiring point cloud data of a target detection area on a railway vehicle; according to the point cloud data, extracting an installation reference surface of a hexagon bolt in the target detection area and point cloud data of each single part after the installation reference surface is removed; extracting hexagonal bolt geometric features corresponding to the point cloud data of each part; judging whether a part corresponding to the part point cloud data is a hexagon bolt or not according to the geometrical characteristics of the hexagon bolt; if the part corresponding to the part point cloud data is the hexagon bolt, whether the hexagon bolt corresponding to the part point cloud data is loosened or not is detected according to the part point cloud data and the installation reference surface. The method can extract the geometrical characteristics of the bolt through the point cloud data to judge the looseness of the bolt, is more suitable for a vehicle bottom hexagon bolt detection scene where the bolt image texture is not obvious, and facilitates the improvement of the looseness fault detection accuracy of the hexagon bolt.

Description

Technical Field

[0001] The present invention relates to the technical field of rail vehicle fault detection, and in particular to a method, a device and a terminal for detecting loosening of hexagonal bolts on a rail vehicle. Background Art

[0002] The tightness of the underbody bolts of rail vehicles is crucial to the safety of vehicle operation. At present, manual inspection is the main inspection method used in the operation and maintenance of rail vehicles. Manual inspection relies on visual identification of anti-loosening marks to judge the tightness of the underbody bolts, which is easily affected by the status of personnel, trench environment, observation angle, etc., and is also prone to missed inspections.

[0003] As inspection work develops towards refinement and intelligence, railway bureaus and subway companies are exploring the use of intelligent inspection robots to check vehicle bottom faults and bolt tightening status through machine vision to improve inspection efficiency and reliability. Among vehicle bottom fault detection, bolt parts detection accounts for the highest proportion, and hexagonal bolts are the most commonly used among vehicle bottom bolt parts. Therefore, loosening detection of hexagonal bolts on rail vehicles is very important.

[0004] With the rapid development of deep learning algorithms, many researchers have proposed using target detection models such as yolo and fast RCN to detect the bolt position area by collecting RGB images of the vehicle bottom and inputting them into the target detection model. In combination with the part point cloud data, the bolt point cloud is segmented by mapping the pixel coordinates of the bolt position marked on the RGB image. Then, based on the point cloud template, the bolt point cloud is aligned and the distance change between the bolt surface and the installation reference surface is calculated to determine whether the bolt is loose. This method of bolt position detection using RGB images is affected by the accuracy of the target detection model, and the model accuracy is directly affected by factors such as the image quality and number of samples collected in the model training data set. For some situations where the light under the vehicle is dark or the bolt spray color is close to the background color, the effect of bolt recognition based solely on RGB images is not ideal. Summary of the invention

[0005] The embodiments of the present invention provide a method, device and terminal for detecting looseness of hexagonal bolts on rail vehicles, so as to solve the problem of poor recognition of bolt tightening status when the light under the vehicle is dim or the bolt spray color is close to the background color.

[0006] In a first aspect, an embodiment of the present invention provides a method for detecting looseness of a hexagonal bolt on a rail vehicle, comprising:

[0007] Obtain point cloud data of the target detection area on the rail vehicle;

[0008] Extracting the installation reference surface of the hexagonal bolts in the target detection area and the point cloud data of each individual part after removing the installation reference surface according to the point cloud data;

[0009] Extracting geometric features of the hexagonal bolt corresponding to the point cloud data of each part;

[0010] According to the geometric features of the hexagonal bolt, determining whether the part corresponding to the part point cloud data is a hexagonal bolt;

[0011] If the part corresponding to the part point cloud data is a hexagonal bolt, whether the hexagonal bolt corresponding to the part point cloud data is loose is detected based on the part point cloud data and the installation reference surface.

[0012] In a possible implementation, the extracting the hexagonal bolt geometric features corresponding to each of the part point cloud data includes:

[0013] For each of the part point cloud data, extracting the part point cloud boundary of the part point cloud data based on the normal vector estimation;

[0014] Projecting the boundary of the part point cloud along the installation reference plane to obtain the point cloud data of the part point cloud data projected on the boundary of the installation reference plane;

[0015] Performing straight edge detection and extraction on the boundary projection point cloud data to obtain a straight edge set corresponding to the boundary projection point cloud data;

[0016] Determine, according to the centroid coordinates of the boundary projection point cloud data and the straight edge set, a perpendicular vector from the centroid of the boundary projection point cloud data to each straight edge in the straight edge set;

[0017] The angles between each of the vertical vectors and other vertical vectors other than the vertical vector are calculated to obtain a set of angles corresponding to the part point cloud data as the hexagonal bolt geometric features corresponding to the part point cloud data.

[0018] In a possible implementation, judging whether the part corresponding to the part point cloud data is a hexagonal bolt according to the geometric features of the hexagonal bolt includes:

[0019] Arrange the angles in the angle set according to a set order, and determine whether there is a preset angle sequence in the arranged angle sequence;

[0020] If a preset angle sequence exists in the arranged angle sequence, it is determined that the part corresponding to the part point cloud data is a hexagonal bolt;

[0021] If the preset angle sequence does not exist in the arranged angle sequence, it is determined that the part corresponding to the part point cloud data is another type of part.

[0022] In a possible implementation, the detecting whether the hexagonal bolt corresponding to the part point cloud data is loose according to the part point cloud data and the installation reference surface includes:

[0023] According to the part point cloud data, the circumscribed circle diameter of the boundary projection of the hexagonal bolt corresponding to the part point cloud data is calculated;

[0024] Determine the hexagonal bolt specification corresponding to the part point cloud data according to the circumscribed circle diameter;

[0025] Determining a bolt loosening threshold according to the hexagonal bolt specifications;

[0026] According to the part point cloud data, determining the nut point cloud data of the hexagonal bolt corresponding to the part point cloud data;

[0027] Calculate the distance from the centroid coordinates of the nut point cloud data to the installation reference surface, and record it as the measured value of the nut surface height of the hexagonal bolt corresponding to the part point cloud data;

[0028] The measured value of the nut surface height is compared with the bolt loosening threshold, and whether the hexagonal bolt corresponding to the part point cloud data is loose is determined according to the comparison result.

[0029] In a possible implementation, the step of calculating the circumscribed circle diameter of the boundary projection of the hexagonal bolt corresponding to the part point cloud data according to the part point cloud data includes:

[0030] Determine, according to the angle set corresponding to the part point cloud data, each adjacent straight edge in the straight edge set corresponding to the part point cloud data;

[0031] Determine at least three vertices of the boundary projection of the hexagonal bolt corresponding to the part point cloud data according to each adjacent straight edge;

[0032] According to the at least three vertices, the diameter of the circumscribed circle of the boundary projection of the hexagonal bolt corresponding to the part point cloud data is calculated.

[0033] In a possible implementation, calculating the distance from the centroid coordinates of the nut point cloud data to the installation reference surface, and recording it as the measured value of the nut surface height of the hexagonal bolt corresponding to the part point cloud data, includes:

[0034] according to Calculating the distance from the centroid coordinates of the nut point cloud data to the installation reference surface;

[0035] Among them, d is the distance from the centroid coordinates of the nut point cloud data to the installation reference plane, recorded as the measured value of the nut surface height of the hexagonal bolt corresponding to the part point cloud data, (x, y, z) is the centroid coordinates of the nut point cloud data, and A, B, C, and D are the plane model parameters of the installation reference plane.

[0036] In a possible implementation, extracting the installation reference surface of the hexagonal bolt in the target detection area and the point cloud data of each individual part after excluding the installation reference surface according to the point cloud data includes:

[0037] Performing plane fitting on the point cloud data to extract the installation reference plane of the hexagonal bolts in the target detection area;

[0038] Performing convex hull calculation on the point cloud data based on the installation reference plane to obtain all part point cloud data after removing the installation reference plane;

[0039] Point cloud clustering and segmentation are performed on all the part point cloud data to obtain individual part point cloud data.

[0040] In a second aspect, an embodiment of the present invention provides a device for detecting loosening of hexagonal bolts on a rail vehicle, comprising:

[0041] An acquisition module, used to acquire point cloud data of a target detection area on a rail vehicle;

[0042] A first processing module is used to extract the installation reference surface of the hexagonal bolts in the target detection area and the point cloud data of each individual part after eliminating the installation reference surface according to the point cloud data;

[0043] The second processing module is used to extract the geometric features of the hexagonal bolts corresponding to the point cloud data of each part;

[0044] A judgment module, used for judging whether the part corresponding to the part point cloud data is a hexagonal bolt according to the geometric features of the hexagonal bolt;

[0045] The detection module is used to detect whether the hexagonal bolt corresponding to the part point cloud data is loose according to the part point cloud data and the installation reference surface if the part corresponding to the part point cloud data is a hexagonal bolt.

[0046] In a third aspect, an embodiment of the present invention provides a terminal, comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to perform the steps of the method described in the first aspect or any possible implementation of the first aspect.

[0047] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect or any possible implementation method of the first aspect are implemented.

[0048] The embodiment of the present invention provides a method, device and terminal for detecting the loosening of hexagonal bolts on a rail vehicle, by acquiring point cloud data of a target detection area on a rail vehicle, and extracting the installation reference surface of the hexagonal bolts in the target detection area and the point cloud data of each individual part after the installation reference surface is removed according to the point cloud data, and then extracting the geometric features of the hexagonal bolts corresponding to each part point cloud data, and then judging whether the part corresponding to the part point cloud data is a hexagonal bolt according to the geometric features of the hexagonal bolts, and in the case where the part corresponding to the part point cloud data is a hexagonal bolt, judging whether the hexagonal bolt corresponding to the part point cloud data is loose according to the part point cloud data and the installation reference surface. Thus, the geometric features of the bolts themselves are extracted by the point cloud data to judge whether the bolts are loose, and the detection effect of the target under the vehicle light and the two-dimensional RGB image is not dependent, and the point cloud matching template does not need to be made. It is more suitable for the undercarriage hexagonal bolt detection scene where the bolt image texture is not obvious, which helps to improve the accuracy of the detection of the loosening fault of the hexagonal bolts. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0050] Figure 1 It is a flow chart of the implementation of the method for detecting looseness of hexagonal bolts on a rail vehicle provided by an embodiment of the present invention;

[0051] Figure 2 is a schematic diagram of the bolt installation state of the target detection area provided by an embodiment of the present invention;

[0052] Figure 3 It is a schematic diagram of projecting the boundary of a part point cloud along the installation reference plane provided by an embodiment of the present invention;

[0053] Figure 4 is a schematic diagram of a vertical vector provided by an embodiment of the present invention;

[0054] Figure 5 is a schematic diagram of local geometric features of the hexagonal bolt geometric features provided by an embodiment of the present invention;

[0055] Figure 6is a schematic diagram of the dimensions of the hexagonal bolts provided in an embodiment of the present invention;

[0056] Figure 7 It is a schematic diagram of the boundary projection and the circumscribed circle of the hexagonal bolt corresponding to the part point cloud data provided by an embodiment of the present invention;

[0057] Figure 8 The nut surface A provided in the embodiment of the present invention 1 B 1 C 1 D 1 E 1 F 1 Schematic diagram of the distance to the bolt installation reference surface;

[0058] Fig. 9 is the target area point cloud P provided by the embodiment of the present invention 1 Schematic diagram of

[0059] Fig.10 is a schematic diagram of a single part point cloud obtained by clustering segmentation provided by an embodiment of the present invention;

[0060] Fig.11 It is a schematic diagram of a boundary projection point cloud BC1 obtained by projecting a point cloud boundary onto an installation reference surface of a bolt provided by an embodiment of the present invention;

[0061] Fig.12 It is a structural schematic diagram of a hexagonal bolt loosening detection device on a rail vehicle provided by an embodiment of the present invention;

[0062] Fig.13 is a schematic diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0063] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0064] In order to make the purpose, technical solutions and advantages of the present invention more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0065] See also Figure 1 , which shows a flow chart of the implementation of the method for detecting loose hexagonal bolts on a rail vehicle provided by an embodiment of the present invention, and is described in detail as follows:

[0066] In step 101, point cloud data of a target detection area on a rail vehicle is obtained.

[0067] For example, a structured light camera, a laser radar, a binocular camera, etc. can be used to obtain point cloud data of a target detection area on a rail vehicle including hexagonal bolts to be detected. The obtained point cloud data can be recorded as a point set P = {p 1 ,p 1 ,…,p n}. The i-th point p in the point cloud data P i The three-dimensional coordinate value can be recorded as P i , P i =g(x i ,y i ,z i ). Where i = 1, 2, ..., n, n is the total number of points in the point cloud data, x i ,y i ,z i ∈R.

[0068] Optionally, in order to remove noise in the process of acquiring point cloud data and reduce the amount of subsequent calculations, the acquired point cloud data may be preprocessed.

[0069] For example, during preprocessing, the point cloud data P may be first subjected to through-filtering to obtain the through-filtered point cloud data P 1 , in order to remove unnecessary point cloud background and outliers that are not within the filter range. Among them, during straight-through filtering, the filter range can be set along the directions of the three coordinate axes of x, y, and z as needed. For example, assuming that the background part in the target detection area is 10 meters away from the camera, and parts such as hexagonal bolts in the target detection area are 1 meter away from the camera, the filter range along the z coordinate axis can be set to 0 to 2 meters to retain points within 0 to 2 meters in the z coordinate axis direction and remove points beyond 2 meters in the z coordinate axis direction.

[0070] After direct filtering, the point cloud data P 1 Perform statistical filtering to obtain the point cloud data P after statistical filtering 2 , to further remove outliers in the point cloud data. When statistical filtering is performed, the point p can be calculated i The average distance μ of the nearest d points is set, and the standard deviation range σ and standard deviation multiple t of the distance are set. The outliers whose distance values ​​are not within the filter interval (μ-σt, μ+σt) are removed to obtain the point cloud data P after statistical filtering. 2 .

[0071] After statistical filtering, the point cloud data P 2 The voxel filtering method is used to simplify and realize point cloud downsampling, so as to reduce the computational complexity of point cloud data processing without destroying the geometric structure of the point cloud itself.2 The maximum and minimum values ​​in the x, y, and z coordinate axes and the set grid side lengths affect the point cloud data P 2 Divide the voxel grid and calculate the centroid of each voxel. Select the point closest to the centroid to replace all points in the voxel grid to obtain the simplified point cloud data P 3 .

[0072] It should be noted that, according to actual needs, the point cloud data P can be subjected to through-filtering, statistical filtering and voxel filtering in the above order, or one or two of through-filtering, statistical filtering and voxel filtering can be used to pre-process the point cloud data P. Subsequent processing can be performed based on the pre-processed point cloud data.

[0073] In step 102, the installation reference surface of the hexagonal bolts in the target detection area and the point cloud data of each individual part after the installation reference surface is removed are extracted based on the point cloud data.

[0074] Exemplarily, extracting the installation reference surface of the hexagonal bolt in the target detection area and the point cloud data of each individual part after excluding the installation reference surface according to the point cloud data may include:

[0075] Perform plane fitting on the point cloud data to extract the installation reference surface of the hexagonal bolts in the target detection area. Perform convex hull calculation on the point cloud data based on the installation reference surface to obtain the point cloud data of all parts after removing the installation reference surface. Perform point cloud clustering and segmentation on all parts point cloud data to obtain the point cloud data of each individual part.

[0076] Among them, Figure 2 As shown in the figure, since the area of ​​the bolt installation reference surface in the selected target detection area is usually larger than the surface area of ​​the bolts and other types of parts installed on the reference surface, the installation reference surface can be extracted by plane fitting. Then, the point cloud data of all parts after eliminating the installation reference surface is obtained by convex hull calculation, and the point cloud data of each individual part is obtained by performing point cloud clustering and segmentation on all part point cloud data.

[0077] For example, the installation reference surface of the bolt can be extracted using the Random Sample Consensus (RANSAC) algorithm, which iteratively estimates mathematical model parameters from a set of observation data sets, and records the data that meet the assumed model parameters as inliers, and the remaining points as outliers.

[0078] When using RANSAC to extract the installation reference surface, you can first establish the plane model to be estimated (that is, the plane model of the installation reference surface):

[0079] Ax+By+Cz+D=0;

[0080] Among them, A, B, C, and D are the parameters to be estimated of the plane model.

[0081] Then from the point cloud data P 3 Randomly select three non-collinear points as the minimum sample set, calculate the values ​​of the estimated parameters A, B, C, and D of the plane model, and calculate the point cloud data P under this value. 3 The distance from all other points in the plane to this plane is the error of each point under this value. Mark the points whose errors are less than the set threshold as inliers, and count the number of inliers under this value of the parameter to be estimated. Continue to perform the above two steps. If the number of inliers in the current plane model is greater than the maximum number of inliers that have been saved, update the value of the parameter to be estimated in the plane model, and keep the value of the parameter to be estimated that always has the largest number of inliers. Repeat this process and iterate continuously until the point cloud data P is found. 3 The plane model containing the most internal points is determined as the installation reference plane of the bolt.

[0082] Optionally, the least squares (LS) method can be used to achieve plane fitting by minimizing the sum of squared errors. Alternatively, the Hough transform method can be used to randomly select m points from the point cloud data, convert the coordinates of these points into Hough space, and vote to determine the planes where these points are located. Alternatively, the maximum likelihood estimation, the minimum median square method, the singular value decomposition method, the Lagrange multiplier method, the principal component analysis method, etc. can be used to achieve plane fitting, which is not limited in this embodiment.

[0083] For example, the convex hull of the point cloud can be generated by using the QuickHull algorithm for the extracted installation reference surface, and the point cloud data P containing all the remaining parts after the installation reference surface is removed is obtained by further segmentation. 4 .

[0084] Among them, the commonly used algorithms for calculating the convex hull of a multidimensional space point set include the incremental algorithm, the divide and conquer algorithm, and the quick packing algorithm. The incremental algorithm is an intuitive convex hull calculation method. Its core idea is to start from a small convex hull, gradually add points and update the convex hull. It is efficient in low-dimensional space, but may encounter inefficiency in high-dimensional space. The divide and conquer algorithm divides the point set into smaller subsets, calculates the convex hull separately, and then merges these small convex hulls into the final convex hull. This method is suitable for any dimensional situation, including high-dimensional space. Among them, the QuickHull algorithm is an algorithm based on the divide and conquer idea. It recursively divides the point set into two subsets, and then finds a point on one side of a straight line in each subset as a point on the convex hull, and finally obtains the convex hull. The time complexity of this algorithm is O(nlogn), where n is the number of points. The quick packing algorithm selects the pole and constructs a local convex hull around it, and then gradually merges these local convex hulls to form a global convex hull. This is currently recognized as one of the fastest convex hull calculation methods. According to actual needs, you can choose a suitable method to calculate the convex hull.

[0085] For example, since the point cloud data obtained by the measuring camera is an unordered point cloud, there is no topological relationship between the points, and it is impossible to directly search and find the neighborhood data points, and it is also impossible to directly segment the single part point cloud data. 4 Create a kd tree index for fast search and accelerated calculation of neighboring points in the point cloud. After creating a kd tree index, you can search by specifying the radius or the number of neighboring points. 4 Perform Euclidean distance clustering segmentation, Manhattan distance clustering segmentation, etc. to segment individual part point cloud data. The segmented individual part point cloud data can be used as point cloud set Cloud = {C 1 ,C 2 ,…C q} indicates that the segmented single part point cloud data includes a single bolt point cloud and other types of part point clouds.

[0086] Optionally, point cloud clustering and segmentation can also be performed by using a density-based clustering method or a geometric feature-based segmentation method, which is not limited in this embodiment. Among them, the density-based clustering method calculates the number of points within a unit volume, and a higher density indicates that the point cloud data is denser in this area. The typical density clustering algorithm is the DBSCAN algorithm. The geometric feature-based segmentation method mainly performs segmentation by calculating the geometric features of the point cloud, such as the normal, curvature, and curvature change of the point. Commonly used algorithms include segmentation based on curvature threshold, segmentation based on region growing, and the like.

[0087] In step 103, the geometric features of the hexagonal bolt corresponding to each part point cloud data are extracted.

[0088] In one embodiment, extracting the hexagonal bolt geometric features corresponding to each part point cloud data may include: for each part point cloud data, extracting the part point cloud boundary of the part point cloud data based on the normal vector estimation. Projecting the part point cloud boundary along the installation reference plane to obtain the boundary projection point cloud data of the part point cloud data on the installation reference plane. Performing straight edge detection and extraction on the boundary projection point cloud data to obtain a straight edge set corresponding to the boundary projection point cloud data. Determining the perpendicular vector from the centroid of the boundary projection point cloud data to each straight edge in the straight edge set based on the centroid coordinates of the boundary projection point cloud data. Calculating the angle between each perpendicular vector and each other perpendicular vector other than the perpendicular vector to obtain the angle set corresponding to the part point cloud data as the hexagonal bolt geometric features corresponding to the part point cloud data.

[0089] like Figure 3 As shown in the figure, in order to extract the geometric features of the hexagonal bolt in each part point cloud data, the point cloud set Cloud is traversed and any segmented part point cloud data C is i (i=1,2,…,q) Estimate the normal vector, extract the point cloud C based on the normal vector estimation and point cloud data i The surface boundary of the part (i.e., the part point cloud boundary) is projected onto the mounting reference plane along the normal vector of the mounting reference plane. i The plane projection of the surface boundary of on the installation reference plane is recorded as point cloud C b (i.e. boundary projection point cloud data).

[0090] Then we can use the random sampling consensus algorithm to project the boundary point cloud data C based on the straight line model. b Detect and extract the straight edges in the image, and save all the estimated straight line model parameters until C b After removing the detected straight edges, the number of remaining points is less than the set threshold. The extracted straight edge set can be recorded as Line = {L 1 ,L 2 ,…Lp}, where p is C b Extract the total number of straight edges from

[0091] Among them, the straight line model to be estimated is:

[0092] (xa) / u=(yb) / v=(zc) / w;

[0093] Among them, (a, b, c) are the coordinate values ​​of a point on the line, and (u, v, w) is the direction vector of the line.

[0094] Project the point cloud data C according to the boundary b The average value of the coordinates of all points in the boundary projection point cloud data C is calculated. bThe centroid coordinates P cent =(∑x j / q,∑y j / q,∑z j / q), through the center of mass P cent Draw a perpendicular line to each straight edge in the straight edge set Line, and calculate the perpendicular line equation to obtain the direction vector of the perpendicular line (i.e., the perpendicular line vector), which is recorded as the set like Figure 4 As shown, p is the total number of perpendicular lines corresponding to the straight edge set Line. The direction vector of each perpendicular line is determined by the center of mass P cent Point to the perpendicular direction. There is a one-to-one correspondence between the direction vector of the perpendicular bisector and the straight edges extracted from the straight edge set Line.

[0095] Iterating over a collection Calculate each perpendicular vector With other vectors The angle θ is calculated as follows:

[0096]

[0097] The value range of θ is [0,π], 1≤i,j≤p. Since the adjacent relationship of each straight edge in the set Line has not been confirmed, the adjacent edges of each edge can be determined by calculating the angle and positive and negative directions between the corresponding vertical vectors. For example, the boundary projection point cloud data C b The normal vector direction is the rotation axis, with the vector As a benchmark, according to the right-hand rule, the set Sort the vectors in and other vectors Perform a cross multiplication operation to obtain the vector product vector If the angle between the normal vector of Cb and θ is less than 90°, θ is considered positive, otherwise, θ is negative. The angle with itself is 0°.

[0098] Among them, since the projection of the shape of the hexagonal bolt nut on the installation reference plane is approximately a regular hexagon, its perpendicular vector set is The theoretical value of the angle between the vectors in is β∈{-180°, -120°, -60°, 0°, 60°, 120°, 180°}. Due to the measurement and calculation errors, the limit value of the angle error can be determined as ε through experiments. For the angle θ, if θ∈[β-ε, β+ε] interval, assign the theoretical value β to the corresponding angle θ, and assign the vector The corresponding updated angle θ is saved to the new angle set Ei={θ1 ,θ 2 ,…,θ h}, where h is a vector With Collection The number of angles between all vectors in the point cloud data. Thus, the geometric features of the hexagonal bolts in each single part point cloud data are characterized by the angles in the angle set.

[0099] In step 104, it is determined whether the part corresponding to the part point cloud data is a hexagonal bolt based on the geometric features of the hexagonal bolt.

[0100] In one embodiment, judging whether the part corresponding to the part point cloud data is a hexagonal bolt according to the geometric features of the hexagonal bolt may include: arranging the angles in the angle set according to a set order, and judging whether there is a preset angle sequence in the arranged angle sequence. If there is a preset angle sequence in the arranged angle sequence, it is determined that the part corresponding to the part point cloud data is a hexagonal bolt. If there is no preset angle sequence in the arranged angle sequence, it is determined that the part corresponding to the part point cloud data is other parts.

[0101] Among them, since the projection of the shape of the hexagonal bolt nut on the installation reference plane is approximately a regular hexagon, its perpendicular vector set is The theoretical value of the angle between the vectors in β∈{-180°, -120°, -60°, 0°, 60°, 120°, 180°}. If the part corresponding to the part point cloud data is a hexagonal bolt, then after arranging the angles in the corresponding angle set in a certain order (i.e., the set order), they should be the same as the angles corresponding to the regular hexagon. Conversely, if the angles in the angle set are arranged in the set order and are different from the angles corresponding to the regular hexagon, it means that the part corresponding to the part point cloud data is not a hexagonal bolt, but other types of parts.

[0102] For example, Figure 5 As shown, the angles in the angle set Ei can be arranged in ascending order, and then it is detected whether there is a sequence of {-120°, -60°, 0°, 60°} and {-60°, 0°, 60°, 120°}. If so, it is determined that the local shape of the part point cloud data satisfies the local geometric feature that the three angles of the four adjacent straight sides in the regular hexagon are 120°, so it is determined that the part type corresponding to the part point cloud data is a hexagonal bolt. If all angle sets Ei are traversed, i , if the above sequence does not exist, it is judged that the part type corresponding to the part point cloud data is other types of parts other than bolts.

[0103] For example, the angle set E can also be iThe angles in the image are arranged in descending order or other order, as long as the corresponding preset angle sequence is changed during detection, which is not limited in this embodiment.

[0104] In step 105, if the part corresponding to the part point cloud data is a hexagonal bolt, whether the hexagonal bolt corresponding to the part point cloud data is loose is detected based on the part point cloud data and the installation reference surface.

[0105] In one embodiment, detecting whether the hexagonal bolt corresponding to the part point cloud data is loose according to the part point cloud data and the installation reference surface may include:

[0106] According to the part point cloud data, calculate the circumscribed circle diameter of the boundary projection of the hexagonal bolt corresponding to the part point cloud data. According to the circumscribed circle diameter, determine the specification of the hexagonal bolt corresponding to the part point cloud data. Determine the bolt loosening threshold according to the hexagonal bolt specification. According to the part point cloud data, determine the nut point cloud data of the hexagonal bolt corresponding to the part point cloud data. Calculate the distance from the centroid coordinates of the nut point cloud data to the installation reference surface, and record it as the measured value of the nut surface height of the hexagonal bolt corresponding to the part point cloud data. Compare the measured value of the nut surface height with the bolt loosening threshold, and determine whether the hexagonal bolt corresponding to the part point cloud data is loose based on the comparison result.

[0107] Combination Figure 6 As shown, the theoretical value range of the nut diagonal width e and nut surface height k of the hexagonal bolts in the target detection area (e.g., the bottom of the vehicle to be detected) can be obtained in advance according to the drawing of the parts of the rail vehicle model to be detected. Among them, the corresponding bolt loosening threshold can be pre-set according to the theoretical value of the nut surface height k, and when the nut diagonal width e is different, it means that the specifications of the hexagonal bolts are different, and the bolt loosening threshold corresponding to the nut surface height k is also different.

[0108] For the part point cloud data determined to be a hexagonal bolt type part, considering that the nut surface of the hexagonal bolt is generally parallel to the installation reference plane, the boundary projection point cloud data obtained after the part point cloud boundary of the hexagonal bolt is projected onto the bolt installation reference plane is basically the same as the nut surface of the hexagonal bolt. Therefore, the circumscribed circle diameter of the boundary projection of the hexagonal bolt corresponding to the part point cloud data can be calculated. The circumscribed circle diameter corresponds to the diagonal width of the nut of the hexagonal bolt, and the specification of the hexagonal bolt can be automatically determined based on the circumscribed circle diameter. Then, the corresponding bolt loosening threshold is determined based on the hexagonal bolt specification, which is used to compare with the measured value of the nut surface height obtained by point cloud measurement to determine whether the bolt is loose.

[0109] Among them, referring to step 103, the part point cloud boundary of the part point cloud data can be extracted based on the normal vector estimation, and the part point cloud boundary can be projected along the installation reference plane to obtain the boundary projection point cloud data of the part point cloud data on the installation reference plane, that is, the boundary projection of the hexagonal bolt corresponding to the part point cloud data.

[0110] In one embodiment, calculating the circumscribed circle diameter of the boundary projection of the hexagonal bolt corresponding to the part point cloud data according to the part point cloud data may include:

[0111] According to the angle set corresponding to the part point cloud data, each adjacent straight edge in the straight edge set corresponding to the part point cloud data is determined. According to each adjacent straight edge, at least three vertices of the boundary projection of the hexagonal bolt corresponding to the part point cloud data are determined. According to the at least three vertices, the circumscribed circle diameter of the boundary projection of the hexagonal bolt corresponding to the part point cloud data is calculated.

[0112] Combination Figure 7 As shown in the figure, for the part point cloud data determined to be a hexagonal bolt type part, for its corresponding boundary projection point cloud data, it can be found in its angle set E i Find the sequences {-120°, -60°, 0°, 60°} and {-60°, 0°, 60°, 120°} in the corresponding line set Line, calculate the coordinates of the intersection of two adjacent straight edges, and get the coordinates of the three vertices of the boundary projection of the hexagonal bolt. Through these three vertices, the equation of the circumscribed circle of the boundary projection of the hexagon can be fitted, and then a set of circumscribed circle center coordinates and circumscribed circle diameter can be calculated. Among them, the circumscribed circle diameter corresponds to the diagonal width of the nut of the hexagonal bolt.

[0113] In order to more accurately determine the specifications of the hexagonal bolts, you can traverse all the angle sets E i Multiple sets of circumscribed circle center coordinates and circumscribed circle diameters are obtained. The average value and standard deviation are calculated for the multiple sets of circumscribed circle center coordinates and circumscribed circle diameters, and outliers are removed. The result of calculating the average value of the circumscribed circle diameter after removing the outliers is used as the final result of the circumscribed circle diameter, and then the hexagonal bolt specification is determined according to the final result of the circumscribed circle diameter, that is, the actual measured value of the diagonal width of the nut of the hexagonal bolt.

[0114] In one embodiment, calculating the distance from the centroid coordinates of the nut point cloud data to the installation reference surface, recorded as the measured value of the nut surface height of the hexagonal bolt corresponding to the part point cloud data, may include:

[0115] according to Calculate the distance from the centroid coordinates of the nut point cloud data to the installation reference surface.

[0116] Among them, d is the distance from the centroid coordinates of the nut point cloud data to the installation reference surface, recorded as the measured value of the nut surface height of the hexagonal bolt corresponding to the part point cloud data, (x, y, z) is the centroid coordinates of the nut point cloud data, and A, B, C, and D are the plane model parameters of the installation reference surface.

[0117] like Figure 8 As shown, after determining the specifications of the hexagonal bolts corresponding to the part point cloud data, the bolt loosening threshold of the hexagonal bolts corresponding to the part point cloud data can be determined. On this basis, as long as the measured value of the nut surface height of the hexagonal bolt corresponding to the part point cloud data can be determined and compared with the corresponding bolt loosening threshold, it can be determined whether the corresponding hexagonal bolt is loose.

[0118] Among them, in order to determine the measured value of the nut surface height of the hexagonal bolt corresponding to the part point cloud data, the hexagonal bolt nut surface A parallel to the installation reference plane can be extracted according to the part point cloud data. 1 B 1 C 1 D 1 E 1 F 1 , and then according to the nut surface A 1 B 1 C 1 D 1 E 1 F 1 The average value of the point cloud data is used to determine the centroid coordinates P of the nut surface point cloud data. bolt =(∑x j / g,∑y j / g,∑z j / g), where g is the total number of points in the point cloud data of the nut surface.

[0119] In order to make the distance from the centroid coordinates of the nut point cloud data to the installation reference surface more accurate, the centroid coordinates P bolt No less than 5 points are selected in the adjacent area, and the distances from these points to the installation reference surface are calculated, and the average value of the distances is calculated as the measured value of the nut surface height. When calculating the distance from the centroid coordinates of the nut point cloud data to the installation reference surface, the plane model parameters A, B, C, and D of the installation reference surface have been determined through the above step 102.

[0120] The calculated measured value of the nut surface height is compared with the bolt loosening threshold of the hexagonal bolt of this size specification. If the distance exceeds the corresponding bolt loosening threshold range, it is determined that the bolt is loose, otherwise it is determined that the bolt is not loose.

[0121] The following is a further description of the above-mentioned method for detecting loose hexagonal bolts on a rail vehicle through a specific embodiment:

[0122] 1. Point cloud preprocessing

[0123] A 240mm*160mm area under a certain high-speed rail vehicle model was selected as the area to be inspected (i.e., the target inspection area). A certain brand of 3D surface structured light camera was used to collect point cloud data of the inspection area and save it as a ply format file. According to the part drawing, the inspection targets in this area include hexagonal bolts and other types of parts.

[0124] Referring to the actual sizes of M12, M16, and M20 hexagonal bolts, combined with multiple actual measurement tests, the threshold ranges for distinguishing the size specifications of the above three types of bolts are set, as shown in Table 1. The nut surface height threshold range is used as the preset loosening threshold range of the bolt, which is used to compare with the actual measured value of the nut surface height obtained by point cloud measurement to determine whether the bolt is loose.

[0125] Table 1 Threshold ranges for three bolt size specifications

[0126] Bolt specifications Nut diagonal width (mm) Nut surface height threshold (mm) M12 19.4-20.6 6.5-8.5 M16 26-27.6 9-11 M20 32.5-34.5 11.5-13.5

[0127] Use VS2022 and PCL library to process the original point cloud P. Use the loadPLYFile function of the PCL library to read the point cloud file and obtain the original point cloud set and its three-dimensional coordinates of the target detection area.

[0128] The getMinMax3D function is used to obtain the maximum value maxPt and the minimum value minPt of the point cloud set, and the PassThrough filter is used to remove the redundant background point cloud, and the following is extracted: Fig. 9 The target area point cloud P shown 1 .

[0129] The StatisticalOutlierRemoval statistical filter is used to calculate the average distance from each point in the point cloud to the K neighboring points. The number of neighborhood points to be queried is set to 50, the distance standard deviation threshold is set to 1 times the standard deviation, and discrete noise points are removed to obtain the filtered point cloud set P. 2 .

[0130] The VoxelGrid voxel grid filter is used to simplify the point cloud data and achieve point cloud downsampling without destroying the geometric structure of the point cloud itself. The grid volume size is set to 0.3mm, and all points in the small grid are replaced by the point closest to the center of gravity in the point cloud data to obtain the point cloud set P. 3 .

[0131] For the point cloud P 3, use the pcl::SACSegmentation class to fit the installation reference surface of the bolt, set the threshold of the distance from the discrete point to the plane model surface to 0.1mm, the maximum number of iterations to 1000, and use the plane model to fit, and get the plane equation to be -0.180143x-0.0360504y+0.98298z-540.069=0.

[0132] Generate a two-dimensional convex hull using the fitted bolt installation reference surface point cloud as the index, specify the height interval range as [10,30], unit mm, generate a polygonal prism model, and segment all the part point cloud sets P contained in the prism model. 4 , used for further segmentation of subsequent single part point cloud.

[0133] For the unordered point cloud set P 4 Establish a kd tree data structure for neighbor search and lookup of point clouds. Use the Euclidean clustering partition function to partition the point cloud set P. 4 To cluster a single part, we select the kd tree nearest neighbor search method and set the cluster search radius to 5 mm. We get three part clusters, which are respectively recorded as point cloud C1, point cloud C2 and point cloud C3. Fig.10 shown.

[0134] The normal vector of point cloud C1 is estimated using the normal vector. The search method selects the kd tree nearest neighbor search algorithm. The search radius is set to 3mm. The normal vector is used as input to calculate the boundary of point cloud C1. The boundary of the point cloud is projected onto the installation reference surface of the bolt to obtain the boundary projection point cloud BC1 after plane projection, as shown in Fig.11 Through the above operations, the preprocessing of the part point cloud for the geometric feature identification of the hexagonal bolt is completed.

[0135] 2. Automatic identification of hexagonal bolts

[0136] Start extracting the geometric features of the hexagonal bolt from the boundary projection point cloud BC1 of the part point cloud. Use the pcl::SACSegmentation class to detect the straight line segments of the boundary projection point cloud BC1 using the straight line model, establish a loop, and set the loop stop condition, that is, after removing the detected straight line points, when the percentage of the remaining points in BC1 to the total number of points is less than 30%, the loop stops. Save all the straight line model parameters extracted from BC1.

[0137] Calculate the coordinates of the centroid of the boundary projection point cloud BC1, and draw a perpendicular line through the centroid to each straight edge extracted from BC1, calculate the parameters of the perpendicular line equation and save them. Define the direction of the direction vector of each perpendicular line from the centroid to the foot of the perpendicular.

[0138] For each vertical line calculated in the previous step, the angle θ between each other vertical line is calculated by traversal, and the sign of the angle θ is determined by the normal vector of the plane where the vertical line is located. The projection of the hexagonal bolt on the bolt installation reference plane is approximately a regular hexagon. The theoretical value of the angle between the straight sides of the regular hexagon is β, then β∈{-180°, -120°, -60°, 0°, 60°, 120°, 180°}. Due to factors such as bolt manufacturing error and measurement error, there will be a certain deviation ε between the measured value of the angle θ between the two vertical lines and the theoretical value β. According to the experiment, the deviation value ε is determined to be 3°. Extract the straight edge corresponding to the angle θ within the range of β±3°, assign the theoretical value β to the corresponding angle θ, and save the calculation result of the updated angle θ to a new set. In this way, each straight edge extracted from the boundary projection point cloud BC1 has a corresponding angle θ set and straight edge set.

[0139] The set of angles θ of each straight edge extracted from the boundary projection point cloud BC1 is arranged in ascending order according to the value, and then the set is checked to see if there is a sequence of {-120°, -60°, 0°, 60°} and {-60°, 0°, 60°, 120°}. If the above sequence exists, it means that the part point cloud shape corresponding to the boundary projection point cloud BC1 conforms to the local geometric features of the bolt hexagon itself, so the part type is judged to be a hexagonal bolt.

[0140] 3. Automatic identification of bolt specifications

[0141] The size specifications of the identified hexagonal bolts are further calculated. According to the numerical order in the angle θ set, the coordinates of the intersection of the four adjacent straight edges in the hexagonal bolt boundary projection point cloud BC1 are calculated as the hexagonal vertices of the hexagonal bolt boundary projection point cloud BC1. Three vertices are extracted from each boundary projection point cloud, fitted into a circle, and the coordinates of the center of the circumscribed circle and the diameter of the circumscribed circle are calculated as the hexagonal circumscribed circle of the bolt point cloud projection. The above calculations are performed for each angle θ set to obtain 6 sets of circumscribed circle diameters.

[0142] The average and standard deviation of all calculated circumscribed circle diameters of BC1 are calculated. There is no calculation result where the circumscribed circle diameter exceeds 2 times the standard deviation. Therefore, the average value of the calculated circumscribed circle diameter of BC1 is 33.46 mm, which is the actual measurement result of the diagonal width of the hexagonal bolt nut. By comparing with the threshold range of the diagonal width of the nut in Table 1, it is determined that the bolt specification of BC1 is an M20 hexagonal bolt.

[0143] 4. Bolt loosening detection

[0144] Calculate the centroid coordinate P of the nut surface of the hexagonal bolt corresponding to point cloud C1 bolt =(∑x j / g,∑y j / g,Σzj / g), g is the total number of points in the point cloud data of the nut surface. bolt Nine points are randomly selected from the adjacent area, and the distances from the points to the installation reference surface of the bolt are calculated respectively. The average value is taken as the measured value of the nut surface height.

[0145] The average value of the distances of the nine points is 12.7 mm, which is within the nut surface height threshold range of the M20 bolt set in Table 1. Therefore, it is determined that the hexagonal bolt corresponding to point cloud C1 is not loose. Similarly, the average values ​​of the circumscribed circle diameters and the measured values ​​of the nut surface height of point clouds C2 and C3 are both within the set nut surface height threshold range of the M20 bolt. Therefore, the hexagonal bolts corresponding to point clouds C2 and C3 are not loose.

[0146] The detection of loose hexagonal bolts on rail vehicles in this embodiment is all based on three-dimensional point cloud data. Through a new hexagonal bolt geometric feature extraction algorithm and combined with point cloud processing technology, hexagonal bolt parts are automatically identified and located, solving the problem of low recognition rate of hexagonal bolt target detection under the vehicle under dark light or unclear texture of the bolt image. Through point cloud measurement data, the distance between the bolt cap surface and the installation reference surface is fitted and calculated to achieve automatic detection of loose bolts, solving the problem that the surface of the bolts under the rail vehicle is affected by dust and rust, the anti-loosening mark line is not clearly painted, and it is difficult to be distinguished and identified by RGB color images. At the same time, there is no need for manual observation and detection one by one, saving labor costs. In addition, this solution does not require the production of point cloud matching templates, which improves the efficiency of model change debugging.

[0147] The embodiment of the present invention obtains point cloud data of a target detection area on a rail vehicle, and extracts the installation reference surface of the hexagonal bolts in the target detection area and the individual part point cloud data after the installation reference surface is removed based on the point cloud data, and then extracts the geometric features of the hexagonal bolts corresponding to each part point cloud data, and then judges whether the part corresponding to the part point cloud data is a hexagonal bolt based on the geometric features of the hexagonal bolt. In the case where the part corresponding to the part point cloud data is a hexagonal bolt, the hexagonal bolt corresponding to the part point cloud data is detected to be loose based on the part point cloud data and the installation reference surface. Thus, the looseness of the bolt is determined by extracting the geometric features of the bolt itself through the point cloud data, which does not rely on the bottom light of the vehicle and the two-dimensional RGB image target detection effect, and does not need to make a point cloud matching template. It is more suitable for the bottom hexagonal bolt detection scene of the vehicle where the bolt image texture is not obvious, which helps to improve the accuracy of the loosening fault detection of the hexagonal bolt.

[0148] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0149] The following is an embodiment of the device of the present invention. For details not described in detail therein, reference may be made to the corresponding method embodiment described above.

[0150] Fig.12 The structural schematic diagram of the hexagonal bolt loosening detection device on a rail vehicle provided by an embodiment of the present invention is shown. For the convenience of description, only the parts related to the embodiment of the present invention are shown, which are described in detail as follows:

[0151] like Fig.12 As shown, the device for detecting loose hexagonal bolts on a rail vehicle includes: an acquisition module 121 , a first processing module 122 , a second processing module 123 , a judgment module 124 and a detection module 125 .

[0152] An acquisition module 121 is used to acquire point cloud data of a target detection area on a rail vehicle;

[0153] A first processing module 122 is used to extract the installation reference surface of the hexagonal bolt in the target detection area and the point cloud data of each individual part after eliminating the installation reference surface according to the point cloud data;

[0154] The second processing module 123 is used to extract the geometric features of the hexagonal bolt corresponding to each of the part point cloud data;

[0155] A judgment module 124, used to judge whether the part corresponding to the part point cloud data is a hexagonal bolt according to the geometric features of the hexagonal bolt;

[0156] The detection module 125 is used to detect whether the hexagonal bolt corresponding to the part point cloud data is loose according to the part point cloud data and the installation reference surface if the part corresponding to the part point cloud data is a hexagonal bolt.

[0157] The embodiment of the present invention obtains point cloud data of a target detection area on a rail vehicle, and extracts the installation reference surface of the hexagonal bolts in the target detection area and the individual part point cloud data after the installation reference surface is removed based on the point cloud data, and then extracts the geometric features of the hexagonal bolts corresponding to each part point cloud data, and then judges whether the part corresponding to the part point cloud data is a hexagonal bolt based on the geometric features of the hexagonal bolt. In the case where the part corresponding to the part point cloud data is a hexagonal bolt, the hexagonal bolt corresponding to the part point cloud data is detected to be loose based on the part point cloud data and the installation reference surface. Thus, the looseness of the bolt is determined by extracting the geometric features of the bolt itself through the point cloud data, which does not rely on the bottom light of the vehicle and the two-dimensional RGB image target detection effect, and does not need to make a point cloud matching template. It is more suitable for the bottom hexagonal bolt detection scene of the vehicle where the bolt image texture is not obvious, which helps to improve the accuracy of the loosening fault detection of the hexagonal bolt.

[0158] In one possible implementation, the second processing module 123 can be used to extract the part point cloud boundary of each part point cloud data based on the normal vector estimation; project the part point cloud boundary along the installation reference plane to obtain the boundary projection point cloud data of the part point cloud data on the installation reference plane; perform straight edge detection and extraction on the boundary projection point cloud data to obtain a straight edge set corresponding to the boundary projection point cloud data; determine the perpendicular vector from the center of mass of the boundary projection point cloud data to each straight edge in the straight edge set according to the center of mass coordinates of the boundary projection point cloud data and the straight edge set; calculate the angle between each of the perpendicular vectors and each other perpendicular vector other than the perpendicular vector to obtain the angle set corresponding to the part point cloud data as the hexagonal bolt geometric feature corresponding to the part point cloud data.

[0159] In a possible implementation, the judgment module 124 can be used to arrange the angles in the angle set according to a set order, and judge whether there is a preset angle sequence in the arranged angle sequence; if there is a preset angle sequence in the arranged angle sequence, it is determined that the part corresponding to the part point cloud data is a hexagonal bolt; if there is no preset angle sequence in the arranged angle sequence, it is determined that the part corresponding to the part point cloud data is another type of part.

[0160] In one possible implementation, the detection module 125 can be used to calculate the circumscribed circle diameter of the boundary projection of the hexagonal bolt corresponding to the part point cloud data based on the part point cloud data; determine the specification of the hexagonal bolt corresponding to the part point cloud data based on the circumscribed circle diameter; determine the bolt loosening threshold based on the hexagonal bolt specification; determine the nut point cloud data of the hexagonal bolt corresponding to the part point cloud data based on the part point cloud data; calculate the distance from the center of mass coordinates of the nut point cloud data to the installation reference plane, and record it as the measured value of the nut surface height of the hexagonal bolt corresponding to the part point cloud data; compare the measured value of the nut surface height with the bolt loosening threshold, and determine whether the hexagonal bolt corresponding to the part point cloud data is loose based on the comparison result.

[0161] In one possible implementation, the detection module 125 can be used to determine each adjacent straight edge in a straight edge set corresponding to the part point cloud data based on the angle set corresponding to the part point cloud data; determine at least three vertices of the boundary projection of the hexagonal bolt corresponding to the part point cloud data based on each adjacent straight edge; and calculate the circumscribed circle diameter of the boundary projection of the hexagonal bolt corresponding to the part point cloud data based on the at least three vertices.

[0162] In a possible implementation, the detection module 125 may be configured to: Calculate the distance from the centroid coordinates of the nut point cloud data to the installation reference plane; wherein, d is the distance from the centroid coordinates of the nut point cloud data to the installation reference plane, recorded as the measured value of the nut surface height of the hexagonal bolt corresponding to the part point cloud data, (x, y, z) is the centroid coordinates of the nut point cloud data, and A, B, C, and D are the plane model parameters of the installation reference plane.

[0163] In one possible implementation, the first processing module 122 can be used to perform plane fitting on the point cloud data to extract the installation reference surface of the hexagonal bolts in the target detection area; perform convex hull calculation on the point cloud data based on the installation reference surface to obtain all part point cloud data after eliminating the installation reference surface; perform point cloud clustering and segmentation on all part point cloud data to obtain each individual part point cloud data.

[0164] Fig.13 is a schematic diagram of a terminal provided by an embodiment of the present invention. Fig.13 As shown, the terminal 13 of this embodiment includes: a processor 130, a memory 131, and a computer program 132 stored in the memory 131 and executable on the processor 130. When the processor 130 executes the computer program 132, the steps in the above-mentioned embodiments of the method for detecting looseness of hexagonal bolts on a rail vehicle are implemented, for example Figure 1 Alternatively, when the processor 130 executes the computer program 132, the functions of each module / unit in the above-mentioned device embodiments are implemented, for example Fig.12 The functions of modules 121 to 125 are shown.

[0165] Exemplarily, the computer program 132 may be divided into one or more modules / units, one or more modules / units are stored in the memory 131, and are executed by the processor 130 to implement the present invention. One or more modules / units may be a series of computer program instruction segments that can implement specific functions, and the instruction segments are used to describe the execution process of the computer program 132 in the terminal 13. For example, the computer program 132 may be divided into Fig.12 Modules 121 to 125 are shown.

[0166] The terminal 13 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal 13 may include, but is not limited to, a processor 130 and a memory 131. Those skilled in the art will appreciate that Fig.13 It is only an example of terminal 13 and does not constitute a limitation on terminal 13. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal may also include input and output devices, network access devices, buses, etc.

[0167] The processor 130 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0168] The memory 131 may be an internal storage unit of the terminal 13, such as a hard disk or memory of the terminal 13. The memory 131 may also be an external storage device of the terminal 13, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal 13. Further, the memory 131 may also include both an internal storage unit of the terminal 13 and an external storage device. The memory 131 is used to store computer programs and other programs and data required by the terminal. The memory 131 may also be used to temporarily store data that has been output or is to be output.

[0169] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0170] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0171] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0172] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0173] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0174] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0175] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned embodiments of the hexagonal bolt loosening detection method on the rail vehicle can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium.

[0176] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A method for detecting loose hexagonal bolts on a rail vehicle, characterized in that: include: Obtain point cloud data of the target detection area on the rail vehicle; Extracting the installation reference surface of the hexagonal bolts in the target detection area and the point cloud data of each individual part after removing the installation reference surface according to the point cloud data; Extracting geometric features of the hexagonal bolt corresponding to the point cloud data of each part; According to the geometric features of the hexagonal bolt, determining whether the part corresponding to the part point cloud data is a hexagonal bolt; If the part corresponding to the part point cloud data is a hexagonal bolt, whether the hexagonal bolt corresponding to the part point cloud data is loose is detected based on the part point cloud data and the installation reference surface.

2. The method for detecting looseness of hexagonal bolts on a rail vehicle according to claim 1, characterized in that: The step of extracting the geometric features of the hexagonal bolts corresponding to each of the part point cloud data comprises: For each of the part point cloud data, extracting the part point cloud boundary of the part point cloud data based on the normal vector estimation; Projecting the boundary of the part point cloud along the installation reference plane to obtain the point cloud data of the part point cloud data projected on the boundary of the installation reference plane; Performing straight edge detection and extraction on the boundary projection point cloud data to obtain a straight edge set corresponding to the boundary projection point cloud data; Determine, according to the centroid coordinates of the boundary projection point cloud data and the straight edge set, a perpendicular vector from the centroid of the boundary projection point cloud data to each straight edge in the straight edge set; The angles between each of the vertical vectors and other vertical vectors other than the vertical vector are calculated to obtain a set of angles corresponding to the part point cloud data as the hexagonal bolt geometric features corresponding to the part point cloud data.

3. The method for detecting looseness of hexagonal bolts on a rail vehicle according to claim 2, characterized in that: Judging whether the part corresponding to the part point cloud data is a hexagonal bolt according to the geometric features of the hexagonal bolt includes: Arrange the angles in the angle set according to a set order, and determine whether there is a preset angle sequence in the arranged angle sequence; If a preset angle sequence exists in the arranged angle sequence, it is determined that the part corresponding to the part point cloud data is a hexagonal bolt; If the preset angle sequence does not exist in the arranged angle sequence, it is determined that the part corresponding to the part point cloud data is another type of part.

4. The method for detecting looseness of hexagonal bolts on a rail vehicle according to claim 1, characterized in that: The detecting whether the hexagonal bolt corresponding to the part point cloud data is loose according to the part point cloud data and the installation reference surface comprises: According to the part point cloud data, the circumscribed circle diameter of the boundary projection of the hexagonal bolt corresponding to the part point cloud data is calculated; Determine the hexagonal bolt specification corresponding to the part point cloud data according to the circumscribed circle diameter; Determining a bolt loosening threshold according to the hexagonal bolt specifications; According to the part point cloud data, determining the nut point cloud data of the hexagonal bolt corresponding to the part point cloud data; Calculate the distance from the centroid coordinates of the nut point cloud data to the installation reference surface, and record it as the measured value of the nut surface height of the hexagonal bolt corresponding to the part point cloud data; The measured value of the nut surface height is compared with the bolt loosening threshold, and whether the hexagonal bolt corresponding to the part point cloud data is loose is determined according to the comparison result.

5. The method for detecting looseness of hexagonal bolts on a rail vehicle according to claim 4, characterized in that: The step of calculating the circumscribed circle diameter of the boundary projection of the hexagonal bolt corresponding to the part point cloud data according to the part point cloud data includes: Determine, according to the angle set corresponding to the part point cloud data, each adjacent straight edge in the straight edge set corresponding to the part point cloud data; Determine at least three vertices of the boundary projection of the hexagonal bolt corresponding to the part point cloud data according to each adjacent straight edge; According to the at least three vertices, the diameter of the circumscribed circle of the boundary projection of the hexagonal bolt corresponding to the part point cloud data is calculated.

6. The method for detecting looseness of hexagonal bolts on a rail vehicle according to claim 4, characterized in that: Calculating the distance from the centroid coordinates of the nut point cloud data to the installation reference surface, and recording it as the measured value of the nut surface height of the hexagonal bolt corresponding to the part point cloud data, includes: according to Calculating the distance from the centroid coordinates of the nut point cloud data to the installation reference surface; Among them, d is the distance from the centroid coordinates of the nut point cloud data to the installation reference plane, recorded as the measured value of the nut surface height of the hexagonal bolt corresponding to the part point cloud data, (x, y, z) is the centroid coordinates of the nut point cloud data, and A, B, C, and D are the plane model parameters of the installation reference plane.

7. The method for detecting looseness of hexagonal bolts on a rail vehicle according to claim 1, characterized in that: Extracting the installation reference surface of the hexagonal bolt in the target detection area and the point cloud data of each individual part after removing the installation reference surface according to the point cloud data, including: Performing plane fitting on the point cloud data to extract the installation reference plane of the hexagonal bolts in the target detection area; Performing convex hull calculation on the point cloud data based on the installation reference plane to obtain all part point cloud data after removing the installation reference plane; Point cloud clustering and segmentation are performed on all the part point cloud data to obtain individual part point cloud data.

8. A device for detecting loose hexagonal bolts on a rail vehicle, characterized in that: include: An acquisition module, used to acquire point cloud data of a target detection area on a rail vehicle; A first processing module is used to extract the installation reference surface of the hexagonal bolts in the target detection area and the point cloud data of each individual part after eliminating the installation reference surface according to the point cloud data; The second processing module is used to extract the geometric features of the hexagonal bolts corresponding to the point cloud data of each part; A judgment module, used for judging whether the part corresponding to the part point cloud data is a hexagonal bolt according to the geometric features of the hexagonal bolt; The detection module is used to detect whether the hexagonal bolt corresponding to the part point cloud data is loose according to the part point cloud data and the installation reference surface if the part corresponding to the part point cloud data is a hexagonal bolt.

9. A terminal, characterized in that: The method comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method as claimed in any one of claims 1 to 7 are implemented.

Images