Truck coupler steel casting point cloud registration method based on deep learning target detection

Abstract

The invention discloses a truck coupler steel casting point cloud registration method based on deep learning target detection. The method comprises the following steps: S1, obtaining local two-dimensional image data and three-dimensional point cloud data of a truck coupler steel casting; s2, a target detection model is established, the target detection model is trained, the trained target detection model is obtained, and the trained target detection model is used for outputting truck coupler steel casting detection frame data; s3, cutting the three-dimensional point cloud data to obtain a point cloud model of the truck coupler steel casting; s4, preprocessing the point cloud model of the truck coupler steel casting; and S5, registering the preprocessed point cloud model of the truck coupler steel casting with a set target point cloud model based on an improved ICP algorithm. According to the invention, the obtained three-dimensional point cloud data is automatically clipped through the target detection model, and the point cloud does not need to be trimmed by manual use of point cloud processing software, so that only the effective point cloud feature region is calculated and processed during point cloud registration; and the complexity and the calculation amount of point cloud processing are considered while a good point cloud registration effect is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection of cast steel parts, and particularly to a point cloud registration method for cast steel parts of freight car couplers based on deep learning object detection. Background Technique

[0002] The freight car coupler is a key component connecting freight car compartments, and its quality and performance are directly related to the running safety and stability of the train. As the main material of the coupler, if there are defects in the cast steel parts, such as cracks, pores, inclusions, etc., it may cause the coupler to fail during use, and then lead to serious safety accidents such as train derailment and collision. Therefore, defect detection of cast steel parts of freight car couplers is an important measure to ensure train operation safety.

[0003] For workpieces such as cast steel parts that may have complex shapes and structures, traditional contact detection may be limited by the shape, resulting in errors or damage to the workpiece surface. With the development of technology, point cloud registration technology has become more and more automated and intelligent, and the point cloud registration technology usually adopts a non-contact scanning method. How to use point cloud registration technology to achieve rapid and accurate detection of cast steel parts is a technical problem to be solved urgently. Summary of the Invention

[0004] The present invention provides a point cloud registration method for cast steel parts of freight car couplers based on deep learning object detection to overcome the technical problems of easy damage and detection errors during the defect detection of cast steel parts.

[0005] To achieve the above object, the technical solution of the present invention is as follows:

[0006] A point cloud registration method for cast steel parts of freight car couplers based on deep learning object detection, the specific steps include:

[0007] S1: Obtain two-dimensional image data and three-dimensional point cloud data of a local part of the cast steel part of the freight car coupler;

[0008] S2: Establish an object detection model based on deep learning, train the object detection model through the two-dimensional image data to obtain a trained object detection model, and the trained object detection model is used to output detection frame data of the cast steel part of the freight car coupler;

[0009] S3: Trim the three-dimensional point cloud data based on the trained object detection model to obtain a point cloud model of the cast steel part of the freight car coupler;

[0010] S4: Preprocess the point cloud model of the cast steel part of the freight car coupler;

[0011] S5: Register the point cloud model of the freight car coupler cast steel part after preprocessing with the set target point cloud model based on the improved ICP algorithm. The improved ICP algorithm improves the ICP algorithm by using the Kd-tree nearest neighbor search algorithm.

[0012] Furthermore, the object detection model is established based on the YOLOv8 network architecture, and the loss function of the object detection model is:

[0013]

[0014] In the formula, b and b gt represent the center point of the predicted bounding box and the true point of the ground truth bounding box respectively, A and A gt represent the predicted bounding box and the ground truth bounding box respectively, ρ 2 (b, b gt ) represents the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box, α represents the weight coefficient, v represents the aspect ratio difference between the two bounding boxes, c represents the length of the diagonal of the minimum bounding rectangle where the predicted bounding box and the ground truth bounding box are located; h gt represents the length of the ground truth bounding box; w and w gt represent the width of the predicted bounding box and the ground truth bounding box respectively.

[0015] Furthermore, the freight car coupler cast steel part detection box data includes confidence, the upper left coordinates of the detection box, the length of the detection box, and the width of the detection box.

[0016] Furthermore, in S4, preprocessing the point cloud model of the freight car coupler cast steel part includes: sequentially filtering the point cloud model of the freight car coupler cast steel part and reducing the point cloud distribution density.

[0017] Furthermore, registering the point cloud model of the freight car coupler cast steel part after preprocessing with the set target point cloud model based on the ICP algorithm improved by the Kd-tree nearest neighbor search algorithm includes:

[0018] 1) Read in the point cloud model of the freight car coupler cast steel part after preprocessing and the set target point cloud model, and configure Kd-tree indexes for the two point cloud models;

[0019] 2) Perform nearest neighbor search on each point of the configured point cloud model of the freight car coupler cast steel part and the target point cloud model to find the nearest point pair for each point in the point cloud model of the freight car coupler cast steel part in the target point cloud model;

[0020] 3) Calculate the rotation matrix and translation vector based on the nearest point pair to obtain the initial transformation matrix;

[0021] 4) Obtain the transformed point cloud model of the freight car coupler cast steel part according to the initial transformation matrix;

[0022] 5) Calculate the distance error between the point cloud model of the transformed freight car coupler cast steel part and the target point cloud model;

[0023] 6) When the distance error converges or reaches the maximum number of iterations, complete the registration of the point cloud model of the freight car coupler cast steel part and the target point cloud model. Otherwise, update the read object in 1) to the point cloud models of the transformed freight car coupler cast steel part and the target point cloud model, and repeat 2)-5) until the point cloud registration is completed.

[0024] Beneficial effects: By establishing an object detection model based on deep learning, the present invention automatically trims the acquired three-dimensional point cloud data to obtain the detection frame data of the freight car coupler cast steel part, eliminating the need for manual complex trimming of the point cloud using point cloud processing software. This enables the calculation and processing to be performed only on the effective point cloud feature regions during point cloud registration, reducing the computational amount and interference of point cloud registration, improving the flexibility of point cloud processing, and taking into account both the complexity and computational amount of point cloud processing while ensuring good results in point cloud registration. At the same time, the ICP algorithm improved by the Kd-tree nearest neighbor search algorithm is used to register the point cloud model of the preprocessed freight car coupler cast steel part and the set target point cloud model, reducing the computational amount and time for finding the nearest neighbor points and improving the efficiency of the final point cloud registration. Description of the Drawings

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0026] Figure 1 It is a flowchart of a point cloud registration method for a freight car coupler cast steel part based on deep learning object detection in the present invention;

[0027] Figure 2 It is a detection result diagram of the detection output of the trained object detection model in the embodiment of the present invention for two-dimensional image data;

[0028] Figure 3 In (a) and (b), they are respectively schematic diagrams before and after trimming the three-dimensional point cloud data by the trained object detection model in the embodiment of the present invention;

[0029] Figure 4 It is a point cloud registration result diagram of the freight car coupler cast steel part in the embodiment of the present invention. Detailed Embodiments

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0031] This embodiment provides a point cloud registration method for cast steel parts of freight car couplers based on deep learning object detection. As Figure 1 shown, the specific steps include:

[0032] S1: Obtain two-dimensional image data and three-dimensional point cloud data of the local part of the cast steel part of the freight car coupler;

[0033] S2: As Figure 2 shown, establish an object detection model based on deep learning, train the object detection model with the two-dimensional image data, and obtain the trained object detection model. The trained object detection model is used to output the detection frame data of the cast steel part of the freight car coupler;

[0034] In a specific embodiment, the object detection model is established based on the YOLOv8 network architecture, and the loss function of the object detection model is:

[0035]

[0036] In the formula, b and b gt respectively represent the center point of the predicted box and the true point of the true box, A and A gt respectively represent the predicted box and the true box, ρ 2 (b, b gt ) represents the Euclidean distance between the center points of the predicted box and the true box, α represents the weight coefficient, v represents the aspect ratio difference between the two boxes, c represents the length of the diagonal of the smallest circumscribed matrix where the predicted box and the true box are located; h gt represents the length of the true box; w and w gt respectively represent the width of the predicted box and the true box.

[0037] Specifically, the registration effect is often affected by the complexity of the actual point cloud and noise. Since the locally captured point cloud model of the freight car coupler cast steel part has the characteristics of complex structure and unclear geometric features, the final point cloud registration effect cannot be guaranteed. Manually using point cloud processing software to trim the original three-dimensional point cloud data and remove the noise and redundant point cloud features in the original three-dimensional point cloud data can achieve the effect of point cloud processing, but it lacks flexibility and is extremely time-consuming and laborious, unable to meet the requirements of practical applications. Therefore, in this embodiment, based on the powerful feature extraction ability and adaptive learning ability of the YOLOv8 network of deep learning, through the learning and training of a large amount of locally two-dimensional image data of freight car coupler cast steel parts in the early stage, after training, it can quickly and accurately process the three-dimensional point cloud data, use the form of a rectangular detection frame to select, and only retain the point cloud data within the range corresponding to the rectangular detection frame, reducing the scale, data volume, and interference of the three-dimensional point cloud model participating in the calculation in the subsequent registration process, focusing more attention on processing effective point cloud features, and finally obtaining the detection frame data of the freight car coupler cast steel part, including confidence, the upper left coordinates of the detection frame, the length of the detection frame, and the width of the detection frame, reducing the interference of useless point cloud features, and improving the registration effect and efficiency of the final point cloud registration method. Using the object detection model based on deep learning to automatically trim the three-dimensional point cloud data enables the point cloud registration to only calculate and process the effective point cloud feature regions, reducing the computational amount of point cloud registration and improving the point cloud registration effect.

[0038] Specifically, in order to further improve the quality and effect of point cloud processing in this embodiment, the loss function of the YOLOv8 network is improved based on the characteristics of the area to be detected in the two-dimensional image of the cast steel part. Specifically, based on the characteristics that the aspect ratios of the features in different areas to be detected in the two-dimensional image of the cast steel part change little and the lengths are basically the same, in the improved loss function, So that the lengths of the features in different areas to be detected in the two-dimensional image of the cast steel part are consistent, thus simplifying the computational amount.

[0039] Through the object detection model, the target categories existing in the image to be detected and the positions of the target categories in the image can be accurately framed. Finally, when trimming the three-dimensional point cloud data, it can not only retain the effective features required for point cloud registration to the greatest extent, but also remove the redundant noise that has little relevance and affects the point cloud registration effect and processing efficiency, making the positioning effect of the detection frame finally output by the target detection network more conducive to subsequent point cloud processing, while reducing the computational complexity and accelerating the convergence speed.

[0040] S3: As Figure 3 shown, based on the trained object detection model, the three-dimensional point cloud data is trimmed to obtain the point cloud model of the freight car coupler cast steel part;

[0041] Specifically, since the two-dimensional image data and three-dimensional point cloud data of the local parts of the freight car coupler cast steel parts are in one-to-one correspondence, the two-dimensional image represents the two-dimensional structure of the cast steel parts, and the three-dimensional point cloud represents the three-dimensional structure of the cast steel parts. The object detection model trained with the two-dimensional image can also limit the range of the three-dimensional point cloud, that is, only the point cloud data within the detection frame range is retained, and finally a point cloud model containing the main geometric effective features can be obtained. Then, this point cloud model is registered with the target point cloud model.

[0042] S4: Preprocess the point cloud model of the freight car coupler cast steel part;

[0043] In a specific embodiment, in S4, preprocessing the point cloud model of the freight car coupler cast steel part includes: sequentially filtering the point cloud model of the freight car coupler cast steel part and reducing the point cloud distribution density.

[0044] Specifically, since the acquired original three-dimensional point cloud data often contains a large amount of point cloud noise caused by the working environment, and the noise has a great impact on the subsequent point cloud registration. Therefore, before starting the registration, first perform point cloud filtering on the original three-dimensional point cloud data to remove the noise randomly and discretely distributed around the effective point cloud model; then perform point cloud reduction on the filtered point cloud model, that is, reduce the overall point distribution density in the point cloud model without damaging the effective features of the point cloud, so as to reduce the calculation amount and improve the point cloud processing efficiency.

[0045] S5: Register the point cloud model of the preprocessed freight car coupler cast steel part with the set target point cloud model based on the improved ICP algorithm, and the improved ICP algorithm improves the ICP algorithm by using the Kd-tree nearest neighbor search algorithm.

[0046] Specifically, since the calculation efficiency of the mainstream ICP point cloud registration algorithm is low, and there will also be a situation where the local optimal registration effect is not ideal. Therefore, in this embodiment, the ICP algorithm improved based on the Kd-tree nearest neighbor search algorithm, and registering the point cloud model of the preprocessed freight car coupler cast steel part with the set target point cloud model based on the ICP algorithm improved based on the Kd-tree nearest neighbor search algorithm includes:

[0047] 1) Read in the point cloud model of the preprocessed freight car coupler cast steel part and the set target point cloud model, and configure Kd-tree indexes for the two point cloud models;

[0048] 2) Perform nearest neighbor search on each point of the point cloud model of the freight car coupler cast steel part and the target point cloud model with the configured Kd-tree indexes to find the nearest point pair for each point in the point cloud model of the freight car coupler cast steel part in the target point cloud model;

[0049] 3) Calculate the rotation matrix and translation vector based on the nearest point pairs to obtain the initial transformation matrix;

[0050] 4) Transform the coordinates of the point cloud model of the freight car coupler steel casting according to the initial transformation matrix to obtain the transformed point cloud model of the freight car coupler steel casting;

[0051] 5) Calculate the distance error between the transformed point cloud model of the freight car coupler steel casting and the target point cloud model;

[0052] 6) When the distance error converges or reaches the maximum number of iterations, complete the registration of the point cloud model of the freight car coupler steel casting and the target point cloud model; otherwise, update the read-in object in 1) to the transformed point cloud model of the freight car coupler steel casting and the target point cloud model, and repeat 2)-5) until the point cloud registration is completed.

[0053] Specifically, the point cloud registration result is often affected by the characteristics of redundant point cloud data, and the registration effect needs to be further improved. For the local point cloud model of the freight car coupler steel casting with a relatively complex structure, the registration effect is poor. To further improve the effect and efficiency of point cloud fine registration, in this embodiment, an ICP algorithm improved based on Kd-tree is used on the basis of the traditional ICP point cloud fine registration algorithm. By using Kd-tree to accelerate the search for corresponding point pairs in the point cloud model of the freight car coupler steel casting and the set target point cloud model, the efficiency of the final point cloud registration is improved.

[0054] Specifically, as Figure 4 shown, the green part is the point cloud model of the freight car coupler steel casting, and the red part is the target point cloud model.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A point cloud registration method for cast steel parts of freight car couplers based on deep learning object detection, characterized in that, The specific steps include: S1: Obtain the two-dimensional image data and three-dimensional point cloud data of the local part of the freight car coupler steel casting; S2: Establish an object detection model based on deep learning, and train the object detection model through the two-dimensional image data to obtain a trained object detection model, which is used to output the detection frame data of the freight car coupler steel casting; S3: Clip the three-dimensional point cloud data based on the trained object detection model to obtain the point cloud model of the freight car coupler steel casting; S4: Preprocess the point cloud model of the freight car coupler steel casting; S5: Register the preprocessed point cloud model of the freight car coupler steel casting with the set target point cloud model based on the improved ICP algorithm, and the improved ICP algorithm improves the ICP algorithm by using the Kd-tree nearest neighbor search algorithm.

2. The method for point cloud registration of the cast steel coupler of a freight car based on deep learning object detection according to claim 1, wherein The object detection model is established based on the YOLOv8 network architecture, and the loss function of the object detection model is: Wherein, b and b gt respectively represent the center point of the prediction box and the true point of the ground truth box, A and A gt respectively represent the prediction box and the ground truth box, ρ 2 (b, b gt ) represents the Euclidean distance between the two center points of the prediction box and the ground truth box, α represents the weight coefficient, v represents the aspect ratio difference between the two boxes, c represents the length of the diagonal of the smallest circumscribed matrix where the prediction box and the ground truth box are located; h gt represents the length of the ground truth box; w and w gt respectively represent the widths of the prediction box and the ground truth box.

3. The method for point cloud registration of the cast steel parts of the freight car coupler based on deep learning object detection according to claim 2, wherein, The detection frame data of the freight car coupler steel casting includes confidence, the coordinates of the upper left corner of the detection frame, the length of the detection frame, and the width of the detection frame.

4. The method for point cloud registration of the cast steel parts of the freight car coupler based on deep learning object detection according to claim 3, wherein In S4, preprocessing the point cloud model of the freight car coupler steel casting includes: sequentially filtering the point cloud model of the freight car coupler steel casting and reducing the point cloud distribution density.

5. The method for point cloud registration of the cast steel coupler of a freight car based on deep learning object detection according to claim 4, wherein Registering the preprocessed point cloud model of the freight car coupler steel casting with the set target point cloud model based on the ICP algorithm improved by the Kd-tree nearest neighbor search algorithm includes: 1) Read in the preprocessed point cloud model of the freight car coupler steel casting and the set target point cloud model, and configure Kd-tree indexes for the two point cloud models; 2) Perform nearest neighbor search on each point of the configured point cloud model of the freight car coupler steel casting and the target point cloud model to find the nearest point pair for each point in the point cloud model of the freight car coupler steel casting in the target point cloud model; 3) Calculate the rotation matrix and translation vector based on the nearest point pair to obtain the initial transformation matrix; 4) Obtain the transformed point cloud model of the freight car coupler steel casting according to the initial transformation matrix; 5) Calculate the distance error between the transformed point cloud model of the freight car coupler steel casting and the target point cloud model; 6) When the distance error converges or reaches the maximum number of iterations, complete the registration of the point cloud model of the freight car coupler steel casting and the target point cloud model, otherwise update the read-in objects in 1) to the transformed point cloud model of the freight car coupler steel casting and the target point cloud model, and repeat 2)-5) until the point cloud registration is completed.

Images