A train parking offset detection method, system and device

Abstract

The present application provides a train deviation detection method, system and device, specifically by constructing a train parking environment 3D point cloud map, selecting a landmark point through the point cloud map, and combining a target detection algorithm to obtain a train deviation landmark point; the robot is navigated to the deviation landmark point for image acquisition task, a change matrix of ICP registration is obtained, registration splicing is carried out combined with the ICP algorithm, target point cloud and point cloud to be registered are obtained; the features of the target point cloud and the point cloud to be registered are extracted, and the NDT algorithm is combined to obtain an NDT transformation matrix; the NDT transformation matrix is used as an ICP initial matrix to register the target point cloud and the point cloud to be registered, and a deviation value is obtained, the deviation value is dynamically calibrated through a mechanical arm, and train parking deviation detection is completed. Starting from the perspective of train parking position, the deviation distance of the train is calculated through the registration algorithm, which replaces the traditional deviation detection method, can greatly save the hardware cost, and improves the flexibility of train parking and the accuracy of parking error calculation.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method, system, and device for detecting train stop deviation. Background Technology

[0002] With the development of SLAM (Simultaneous Localization and Mapping) technology, robots can use onboard sensors to perform self-localization and map building in unknown environments, and achieve navigation based on the generated scene map. Robotic autonomous navigation capabilities are widely used in the field of intelligent inspection, replacing manual labor in complex and high-risk inspection tasks, such as the inspection of rail transit vehicles. Rail transit vehicles require comprehensive maintenance of the train body. Using robots to inspect vehicles can greatly improve maintenance efficiency. However, in scenarios where rail transit parking garages are used for mapping and localization, robot localization is affected by the train's parking position, leading to larger localization errors when inspecting the bottom of the train, and deteriorating the inspection and recognition results.

[0003] Currently, reducing train stopping errors mainly focuses on the braking characteristics of the train itself, researching stopping control algorithms, and achieving a good stopping effect by modifying different braking strategies. For example, installing distance sensors on the train allows for more accurate train position information through signal transmission and reception of reflected signals, thus improving stopping accuracy. While this method improves positioning accuracy during the stopping phase, it requires a large number of sensors, significantly increasing the cost of urban rail transit equipment and making it unsuitable for large-scale rail transit development and application. Another approach is to use fuzzy inference and fuzzy control algorithms in stopping control systems. This involves dividing the stopping phase into multiple segments using inference rules and fuzzy decision-making, predicting stopping accuracy, safety, and comfort indicators for the next moment, and determining the appropriate output. Fuzzy control relies on expert experience to formulate rules; incomplete expert experience can lead to incomplete control rules and coarse results. Furthermore, external parameters constantly change during train operation; without real-time updates to control parameters, the output may not match the actual situation. Stopping accuracy is also affected by system delays, stopping resistance, and other factors. Summary of the Invention

[0004] The present invention proposes a train stop deviation detection method, which can at least solve one of the technical problems in the background art.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A method for detecting train stop deviation specifically includes the following steps: S1. Construct a 3D point cloud map of the site environment; S2. Select inspection points based on the 3D point cloud map to calculate train offset; S3. Correct the inspection points by using target detection algorithms in visual recognition and determine the train deviation marker points; S4. Use a robot to patrol to the train offset marker point, collect images, remove outliers and filter the collected images, obtain the ICP registration transformation matrix from the processed file, and perform registration and stitching in combination with the ICP algorithm to obtain the target point cloud and the point cloud to be registered. S5. Obtain features from the target point cloud and the point cloud to be registered, and combine them with the NDT algorithm to obtain the NDT transformation matrix; S6. Use the NDT transformation matrix as the initial matrix for ICP to register the target point cloud and the point cloud to be registered, and obtain the offset value. S7, Dynamic calibration offset value of robotic arm.

[0006] Furthermore, the method for selecting inspection points in step S2 of the present invention includes the following specific steps: Step 1: Preset the track angle θ: The angle between the track where the train stops and the horizontal direction is defined as θ. Step 1: Preset the track angle θ: The angle between the track where the train stops and the horizontal direction is defined as θ. Step 2: Select image acquisition points: Based on the train structure, select robot vision image acquisition points that meet the requirements; Step 3: Adjust the sampling angle: To eliminate the offset error caused by the shooting angle, the mounting angle of the 3D camera was adjusted to be consistent with the angle θ of the vehicle track; Step 4: Record the inspection points: After image acquisition is completed, the location and its corresponding angle θ are recorded in the 3D point cloud map. This location serves as a marker for calculating train offset. It serves as a reference for subsequent offset calculations and robot navigation and positioning.

[0007] Furthermore, the train offset marker point in step S3 of this invention is updated using a specific method, the steps of which are as follows: S31. The robot navigates to the initial marker point, collects images of the marker point, and trains the dataset. S32. Detect the image using the target algorithm to determine whether renavigation to a new marker point is required; If a marker is detected in the image, its location is recorded as the train offset marker location. ; If no marker component is detected in the image, it indicates that there is a deviation in the current navigation position of the robot. The robot position needs to be corrected, the navigation needs to be reissued, and the process jumps to step S31 until the train offset marker point is selected. S33, Correct the robot's positioning and pose.

[0008] Furthermore, the specific method for training the dataset in step S31 of the present invention is as follows: S311. Navigate the robot to the marker location to collect the dataset, and label the dataset with a certain part of the train (such as a wheel) as the part to be identified. S312. Use the target detection algorithm in visual recognition to train the dataset. In order to ensure that the robot can recognize the labeled parts in the pictures, the label categories are divided into three categories: the first category is the left half of the part, the second category is the whole part, and the third category is the right half of the part. S313. For the images used in the later dynamic calibration of the robotic arm, select the previous one. Each inspection point serves as a dynamic calibration offset point for the robotic arm. The robot is navigated to the calibration point, and a train component is selected as a reference standard at each calibration point. Multiple images of the component are taken, and all images are labeled to train a dataset, which will serve as a reference for subsequent dynamic calibration of the robotic arm.

[0009] Furthermore, the method for obtaining the target point cloud and the point cloud to be registered in step S4 of the present invention is specifically as follows: S41. The robotic arm moves a fixed distance to take pictures, expanding the field of view of the captured images, and an initial matrix for point cloud registration is given. Record the position of the robot arm taking the picture as the center position. The end point moves horizontally left and right to collect data at the middle position. That is, the left side of the photo position And the photo location on the right Take three photos: left, center, and right. The photos are in PLY format and labeled with the given names. , and Save it in the specified path; S42. Read three PLY files and perform outlier removal and filtering on the three PLY files; S43. Obtain the transformation matrix of ICP registration and perform registration stitching in combination with the ICP algorithm; S44. Obtain the target point cloud and the point cloud to be registered.

[0010] Furthermore, the method for obtaining the NDT transformation matrix in step S5 of the present invention is as follows: S51. Extract the point cloud to be registered. and target point cloud The features in the data are used to generate new point clouds to be registered. and target point cloud ; S52. Register the new point cloud to be registered using the NDT algorithm. and target point cloud A transformation matrix is ​​obtained. .

[0011] The NDT algorithm (Normal Distributions Transform) is a classic point cloud registration algorithm. The basic idea is to construct a normal distribution of multidimensional variables based on reference data, and then use optimization methods to find the transformation parameters that maximize the sum of probability densities, thereby achieving the best matching of two laser point cloud data.

[0012] S53. To accelerate convergence, the initial transformation matrix for NDT registration is obtained by traversing initial values, i.e., pre-setting... The interval size is Using the offset values, construct the transformation matrix. ,in This Each of the transformation matrices is used as the initial transformation matrix in the NDT algorithm for registration calculation, resulting in... The transformation matrix corresponding to the minimum registration score is the final transformation matrix for NDT registration. .

[0013] Furthermore, the specific steps for dynamically calibrating the offset value of the robotic arm in step S7 of the present invention are as follows: S71. Use the train stopping offset value obtained in step S6 to perform pose compensation on all inspection points on the map. S72. When the robot performs inspection tasks, at each offset calibration point, it first takes a picture of the selected train component at each calibration point, puts the picture into the target detection network, and verifies the result of component target detection. S73. Combining the two constraints of the detected component probability and the label box being close to the center of the entire image in the results, calculate the pixel coordinate distance between the component and the image center, and use the proportional relationship between the pixel coordinate system and the world coordinate system. The error value is calculated, and the same operation is performed at each calibration point to calculate the average error value, update the calculated train stopping offset value, and perform the inspection task.

[0014] On the other hand, a train stop deviation detection system includes an image acquisition unit, an automatic patrol unit, and a central control unit; the image acquisition unit is a 3D structured light sensor; the automatic patrol unit is a train inspection robot and a SLAM navigation and positioning system; the central control unit includes a robotic arm photo control system and a 3D point cloud image generation system, which are connected to and control the image acquisition unit and the automatic patrol unit. The inspection robot is a wheeled robot with a six-degree-of-freedom robotic arm mounted on its body. The end of the robotic arm is equipped with a 3D structured light sensor. The wheeled robot stops at the set point using the SLAM navigation and positioning algorithm. After it arrives at the set point, the central control unit controls the movement of the robotic arm. The 3D structured light sensor is responsible for collecting image information and generating point cloud files.

[0015] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0016] In summary, this invention constructs a 3D point cloud map of the train parking environment, selects marker points using the point cloud map, and obtains train offset marker points using a target detection algorithm. A robot is navigated to the offset marker points to perform image acquisition, obtaining the ICP registration transformation matrix. The ICP algorithm is then used for registration and stitching to obtain the target point cloud and the point cloud to be registered. Features are extracted from the target point cloud and the point cloud to be registered, and the NDT algorithm is used to obtain the NDT transformation matrix. The NDT transformation matrix is ​​used as the initial matrix for ICP registration of the target point cloud and the point cloud to be registered, obtaining the offset value. The offset value is then dynamically calibrated by a robotic arm to complete the train parking offset detection.

[0017] Compared to traditional methods, it has the following advantages: 1. This invention, starting from the perspective of the train's stopping position, calculates the train's offset distance using a registration algorithm to meet the requirements. Parking deviation recognition, success rate Calculation error accuracy This method replaces methods that rely on the train's braking characteristics and control algorithms to reduce stopping deviation, significantly saving hardware costs and improving the flexibility and accuracy of stopping error calculations.

[0018] 2. This invention addresses the issue of overlap between the target point cloud and the point cloud to be registered during ICP registration. It takes three images by moving a robotic arm and stitches them together at the point cloud level, expanding the field of view of the captured images and increasing the overlap between the target point cloud and the point cloud to be registered, thereby improving the registration success rate.

[0019] 3. This invention improves registration efficiency by setting an initial transformation matrix and traversing the transformation matrix. Furthermore, when combined with a relatively accurate initial transformation matrix, it can improve the accuracy of registration, thereby increasing the correctness of the calculated train stopping offset value.

[0020] 4. This invention combines visual recognition target detection algorithms to correct the robot's navigation pose by detecting marked components. It also uses dynamic calibration of the robotic arm combined with target detection to calibrate the calculated offset value, further improving the accuracy of the train stopping offset value. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the train stop deviation detection process of the present invention; Figure 2 This is a flowchart illustrating the calculation of train stopping offset value according to the present invention. Figure 3 Take three images (left, center, and right) of the marking component for the robotic arm and stitch them together to create a point cloud effect. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0023] To address the problems existing in the prior art, the technical solution adopted by this invention is: to provide a train stopping deviation detection method, the specific steps of which are as follows: S1. Construct a 3D point cloud map of the site environment; Based on the sensors carried by the robot, a relevant SLAM algorithm is selected to generate a 3D point cloud map; S2. Select inspection points based on the 3D point cloud map to calculate train offset; The method for selecting the inspection points for offset calculation is as follows: Step 1: Preset the track angle θ: The angle between the track where the train stops and the horizontal direction is defined as θ. Step 2: Select image acquisition points: Based on the train structure, select image acquisition points that can provide rich component features and are easy for robot vision to recognize. These image acquisition points should include easily identifiable and distinctive components, such as wheels, door edges, or specific markings.

[0024] Step 3: Adjust the sampling angle: To eliminate offset errors caused by the shooting angle, it is essential to ensure that the mounting angle of the 3D camera is consistent with the angle θ of the vehicle track. This means that the optical axis of the camera lens should be parallel to the direction of the vehicle track, so that the features of the components in the image will not experience additional displacement errors due to perspective distortion during shooting.

[0025] Step 4: Record the inspection points: After image acquisition is completed, the location and its corresponding angle θ are recorded in the 3D point cloud map. This location serves as a "marker point for calculating train offset". ", which serves as a reference for subsequent offset calculations and robot navigation and positioning.

[0026] S3. Correct the inspection points by using target detection algorithms in visual recognition and determine the train deviation marker points; The process for determining the location of the train offset marker is as follows: S31. The robot navigates to the initial inspection point, collects images of the marker points, and trains the dataset. S32. Detect the image using the target algorithm to determine whether it is necessary to re-navigate to a new inspection point; If a marker component is detected in the image, the inspection point is recorded as a train offset marker point. ; If no marker component is detected in the image, it indicates that there is a deviation in the current navigation position of the robot. The robot position needs to be corrected, the navigation needs to be reissued, and the process should proceed to step S31 until the train offset marker point is selected.

[0027] S33. Correct the robot's positioning and pose; The specific steps for training the dataset in step S31 are as follows: S311. Navigate the robot to the marker location to collect the dataset, and label the dataset with a certain part of the train (such as a wheel) as the part to be identified. S312. Use the target detection algorithm in visual recognition to train the dataset. In order to ensure that the robot can recognize the labeled parts in the pictures, the label categories are divided into three categories: the first category is the left half of the part, the second category is the whole part, and the third category is the right half of the part. S313. For the images used in the later dynamic calibration of the robotic arm, select the previous one. Each inspection point serves as a dynamic calibration offset point for the robotic arm. The robot is navigated to the calibration point, and a train component is selected as a reference standard at each calibration point. Multiple images of the component are taken, and all images are labeled to train a dataset, which will serve as a reference for subsequent dynamic calibration of the robotic arm.

[0028] The object detection algorithm uses the YOLOv8 algorithm. The YOLOv8 algorithm introduces a new network module called DensePrediction Module (DPM) to directly perform dense prediction based on high-dimensional feature maps. In addition, YOLOv8 also uses a technique called Soft-Gated Skip Connection (SGSC) to dynamically adjust the relationship between different feature maps using gated neurons to capture the features of target objects at different scales. The implementation of YOLOv8 includes data preparation, model building, model training, model evaluation, prediction and localization.

[0029] S4. Use a robot to patrol to the train offset marker point, collect images, remove outliers and filter the collected images, obtain the ICP registration transformation matrix from the processed file, and perform registration and stitching in combination with the ICP algorithm to obtain the target point cloud and the point cloud to be registered. The specific method for obtaining the target point cloud and the point cloud to be registered is as follows: S41. The robotic arm moves a fixed distance to take pictures, expanding the field of view of the captured images, and an initial matrix for point cloud registration is given. Record the position of the robot arm taking the picture as the center position. The end point moves horizontally left and right to collect data at the middle position. That is, the left side of the photo position And the photo location on the right Take three photos: left, center, and right. The photos are in PLY format and labeled with the given names. , and Save it in the specified path; S42. Read three PLY files and perform outlier removal and filtering on the three PLY files; The specific operations for outlier removal and filtering are as follows: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] , and Invalid and abnormal points in the file are then filtered through downsampling, which reduces the error of ICP algorithm registration and stitching, and effectively improves the overlap between the target point cloud, the point cloud to be registered, and the image.

[0030] S43. Obtain the transformation matrix of ICP registration and perform registration stitching in combination with the ICP algorithm; With the 3D camera in the center PLY file obtained from taking a photo Based on the left and right horizontal movement, the resulting PLY file was captured. and Each point cloud is then registered using the ICP algorithm. The ICP algorithm (Iterative Closest Point) is a point cloud registration algorithm used to align the coordinate systems of two or more point clouds. Its principle is to iteratively optimize the matching error between the two point clouds until the error is minimized.

[0031] The initial value of the transformation matrix for ICP registration is based on the distance value dynamically calibrated by the robotic arm. Settings, i.e. and .

[0032] S44. Obtain the target point cloud and the point cloud to be registered; Will and Substitute it into the ICP algorithm and use ICP to obtain and Registration transformation matrix , and Registration transformation matrix Transform the left and right point clouds using a transformation matrix and Unify the coordinates of the three point clouds to the coordinate system of the intermediate point cloud, and stitch them together. Record the stitched point cloud as the target point cloud. Point concentration This represents the coordinates of each point in the target point cloud. , This indicates the number of points in the target point cloud.

[0033] The process of generating the registration PLY file by stitching together point clouds is as described above. The robot navigates to the marker point where the train offset value is calculated. The robotic arm moves a fixed distance to acquire image information. The acquired left, center, and right images are registered using the ICP algorithm to obtain a transformation matrix. This transformation matrix is ​​then determined based on the distance the robotic arm has moved. Set initial values, obtain the transformation matrix using ICP registration, and stitch together the three point clouds. Record the stitched point cloud as the point cloud to be registered. Point concentration This represents the coordinates of each point in the target point cloud. , This indicates the number of points in the target point cloud.

[0034] S5. Obtain features from the target point cloud and the point cloud to be registered, and then obtain the NDT transformation matrix using the NDT algorithm. The specific method for obtaining the NDT transformation matrix is ​​as follows: S51. Extract the point cloud to be registered. and target point cloud The features in the data are used to generate new point clouds to be registered. and target point cloud ; S52. Register the new point cloud to be registered using the NDT algorithm. and target point cloud A transformation matrix is ​​obtained. .

[0035] The NDT algorithm (Normal Distributions Transform) is a classic point cloud registration algorithm. The basic idea is to construct a normal distribution of multidimensional variables based on reference data, and then use optimization methods to find the transformation parameters that maximize the sum of probability densities, thereby achieving the best matching of two laser point cloud data.

[0036] S53. To accelerate convergence, the initial transformation matrix for NDT registration is obtained by traversing initial values, i.e., pre-setting... The interval size is Using the offset values, construct the transformation matrix. ,in This Each of the transformation matrices is used as the initial transformation matrix in the NDT algorithm for registration calculation, resulting in... The transformation matrix corresponding to the minimum registration score is the final transformation matrix for NDT registration. .

[0037] S6. Use the NDT transformation matrix as the initial matrix for ICP to register the target point cloud and the point cloud to be registered, and obtain the offset value. by As the initial transformation matrix for ICP, the point cloud to be registered is used for registration. and target point cloud To obtain the registration score and transformation matrix. .

[0038] Set score threshold The registration score determines whether the registration was successful; if successful, the final ICP registration transformation matrix is ​​obtained. The translation value is the train stopping offset value; if it fails, return to step S2.

[0039] S7 robotic arm dynamic calibration offset value; The train stopping offset value obtained through registration may contain errors. To further improve the accuracy of the offset value, the calculated train offset value can be corrected. By dynamically moving the robotic arm and comparing the images before and after, since the robot and the train are now in the same coordinate system and the included angle is taken into account, and the shooting distance between the camera at the end of the robotic arm and the train is fixed, the captured images have the same resolution. The same axial dimensions can be ignored. Since the camera's intrinsic and extrinsic parameters are known, the ratio between the component's scale in the pixel coordinate system and its scale in the world coordinate system can be determined. ,in, This represents the coordinates of the component in the pixel coordinate system. This represents the coordinates of the component in the world coordinate system. This indicates the conversion relationship between two coordinate systems. Based on the results of the object detection algorithm detecting parts, the distance of the part relative to the image center is obtained. Using the proportional relationship, the error value of the captured part in the world coordinate system is calculated, thereby correcting the offset value.

[0040] The specific steps for dynamically calibrating the offset value of the robotic arm are as follows: S71. Use the train stopping offset value obtained in step S6 to perform pose compensation on all inspection points on the map. This compensation measure aims to correct for any deviations in subsequent navigation points that may result from inaccurate train stopping positions, ensuring that the robot can reach each inspection location more accurately.

[0041] S72. When the robot performs inspection tasks, at each offset calibration point, it first takes a picture of the selected train component at each calibration point, puts the picture into the target detection network, and verifies the result of component target detection. S73. Combining the two constraints of the detected component probability and the label box being close to the center of the entire image in the results, calculate the pixel coordinate distance between the component and the image center, and use the proportional relationship between the pixel coordinate system and the world coordinate system. The error value is calculated, and the same operation is performed at each calibration point to calculate the average error value, update the calculated train stopping offset value, and perform the inspection task.

[0042] Example 1: This embodiment describes the calculation process of a train stop deviation detection method based on 3D point cloud registration using a wheeled inspection robot, as follows: Figure 2 As shown, the specific steps include: Step 1: Construct a 3D point cloud map of the environment. Based on the sensors carried by the robot (LiDAR, IMU, and cameras, etc.), control the robot's movement and select relevant SLAM algorithms, such as FASTER-LIO, ORB-SLAM3, R3LIVE, etc., to generate a 3D point cloud map; Step 2: Select a location where the 3D camera mounted on the robotic arm can capture many features of the train. This section sets up an inspection point for calculating train stopping offset and configures the task of calculating the offset value. A dataset is collected at this point, and wheel components in the images are labeled. The dataset is then trained using an object detection algorithm. The first 20 inspection points are also set as dynamic offset calibration points for the robotic arm. The robot is navigated to these points, and components captured by the camera at each point are selected. The points are positioned so that the captured components are centered in each image. The components in each image are labeled and used in the object detection algorithm for training.

[0043] Step 3: Based on the known camera intrinsic and extrinsic parameters, and since the image capture distance is fixed, calculate the proportional relationship between the two coordinate systems by combining the scale of the component in the pixel coordinate system and the scale of the component in the world coordinate system. Camera intrinsic parameters include focal length and pixel size; extrinsic parameters are the camera's parameters in the world coordinate system, such as the rigid transformation of the camera relative to the vehicle's coordinate system.

[0044] Step 4: The robot navigates to the location. At this location, a 3D camera takes a picture, and the image is input into the target detection network to detect whether the wheel component exists. If it exists, proceed to the next step; otherwise, jump to step two and resend navigation to the inspection point. The task is to calibrate the robot's navigation and positioning errors.

[0045] Step 5: Set the horizontal movement distance of the robotic arm. The robotic arm takes a picture with a 3D camera from the center position, saving the image in PLY format and labeling it "plymiddlefile". Similarly, the robotic arm moves to the left... Two images were taken at different times, and the images were saved in .ply format and labeled as .plyleftfile and .plyrightfile.

[0046] Step 6: Load these three Ply files, remove outliers, and filter these three point clouds at a resolution of 0.01m. Use the PlyMiddlefile point cloud as the target point cloud for ICP registration, and the PlyLeftfile and PlyRightfile point clouds as the point clouds to be registered, respectively. Perform ICP registration on these three point clouds. Set the initial registration matrix. and Substituting these values ​​into the ICP registration algorithm, two transformation matrices are obtained respectively. and The left and right point clouds are transformed using a transformation matrix. and Unify the coordinates to the coordinate system of the intermediate point cloud and stitch the three point clouds together. Figure 3 (a) The robotic arm moves to the left before splicing. Point cloud display diagram (b) shows the point cloud display diagram of the machine at the middle position. Point cloud display diagram (c) shows the robot arm moving to the right before splicing. Point cloud display (d) The final stitched point cloud display.

[0047] Step 7: Check if a file named [name] exists in the specified path. The target point cloud file is used to determine whether the current process belongs to the target point cloud generation stage or the train parking offset calculation stage. If it does not exist The file belongs to the target point cloud generation stage, and the stitched point cloud is called the target point cloud. Save the file in .ply format to the specified path and end the process.

[0048] If it exists The file pertains to the train stopping offset calculation stage, and the stitched point cloud is denoted as the point cloud to be registered. Save it in ply format in the specified path and proceed to step eight.

[0049] Step 8: Load the target point cloud file Extract target point cloud and the point cloud to be matched Features in the point cloud, transferring feature points from the point cloud and Segment the data and generate a new point cloud file. and .

[0050] Step 9: Set the interval size to , There are a total of 20 initial matrices for NDT registration, namely: ,in Substitute these 20 transformation matrices into the NDT algorithm for registration calculation to obtain 20 registration scores. The transformation matrix corresponding to the lowest score is the final transformation matrix for NDT registration. .

[0051] Step 10, with As the initial transformation matrix for ICP, the point cloud to be registered is used for registration. and target point cloud To obtain the registration score and transformation matrix. .

[0052] Step 11: Set the score threshold The registration score determines whether the registration was successful. If the registration was successful, the transformation matrix of the final ICP registration is taken. The translation value in the table is the train stopping offset value. If the registration fails, return to step four.

[0053] Step 12: During the task execution, before taking the task image each time the navigation reaches the offset calibration point, first take an image of the reference component for the robot arm's dynamic calibration at that point. Put the captured image into the target detection network to obtain the detection results. Calculate the pixel distance between the component marked by the label box in the results and the center of the image, using the proportional relationship between the pixel coordinate system and the world coordinate system. The error of the offset value is calculated, and the average of the errors calculated at 20 offset calibration points is taken. The train stopping offset value calculated by the average error is the final train stopping offset value, and the task issued for correction is executed.

[0054] In summary, this invention constructs a 3D point cloud map of the train parking environment, selects marker points using the point cloud map, and obtains train offset marker points using a target detection algorithm. A robot is navigated to the offset marker points to perform image acquisition, obtaining the ICP registration transformation matrix. The ICP algorithm is then used for registration and stitching to obtain the target point cloud and the point cloud to be registered. Features are extracted from the target point cloud and the point cloud to be registered, and the NDT algorithm is used to obtain the NDT transformation matrix. The NDT transformation matrix is ​​used as the initial matrix for ICP registration of the target point cloud and the point cloud to be registered, obtaining the offset value. The offset value is then dynamically calibrated by a robotic arm to complete the train parking offset detection.

[0055] Compared with the prior art, the beneficial effects of this invention are as follows: 1. This invention, starting from the perspective of the train's stopping position, calculates the train's offset distance using a registration algorithm to meet the requirements. Parking deviation recognition, success rate Calculation error accuracy This method replaces methods that rely on the train's braking characteristics and control algorithms to reduce stopping deviation, significantly saving hardware costs and improving the flexibility and accuracy of stopping error calculations.

[0056] 2. This invention addresses the issue of overlap between the target point cloud and the point cloud to be registered during ICP registration. It takes three images by moving a robotic arm and stitches them together at the point cloud level, expanding the field of view of the captured images and increasing the overlap between the target point cloud and the point cloud to be registered, thereby improving the registration success rate.

[0057] 3. This invention improves registration efficiency by setting an initial transformation matrix and traversing the transformation matrix. Furthermore, when combined with a relatively accurate initial transformation matrix, it can improve the accuracy of registration, thereby increasing the correctness of the calculated train stopping offset value.

[0058] 4. This invention combines visual recognition target detection algorithms to correct the robot's navigation pose by detecting marked components. It also uses dynamic calibration of the robotic arm combined with target detection to calibrate the calculated offset value, further improving the accuracy of the train stopping offset value.

[0059] On the other hand, the present invention also discloses a train stop deviation detection system, including an image acquisition unit, an automatic patrol unit, and a central control unit. The image acquisition unit is a 3D structured light sensor; the automatic patrol unit is a train inspection robot and a SLAM navigation and positioning system; the central control unit includes a robotic arm photo-taking control system and a 3D point cloud image generation system.

[0060] The inspection robot is a wheeled robot with a six-degree-of-freedom robotic arm mounted on its body. The end of the robotic arm is equipped with a 3D structured light sensor. The wheeled robot stops at the set point using the SLAM navigation and positioning algorithm. After it arrives at the set point, the central control unit controls the movement of the robotic arm. The 3D structured light sensor is responsible for collecting image information and generating point cloud files.

[0061] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0062] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the train stop deviation detection methods described above.

[0063] It is understood that the systems, devices, and storage media provided in the embodiments of the present invention correspond to the methods provided in the embodiments of the present invention, and the explanations, examples, and beneficial effects of the relevant content can be referred to the corresponding parts of the above methods.

[0064] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0065] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0066] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0067] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting train stop deviation, characterized in that, Specifically, the following steps are included: S1. Construct a 3D point cloud map of the site environment; S2. Select inspection points based on the 3D point cloud map to calculate train offset; S3. Correct the inspection points using target detection algorithms and determine the train deviation marker points; S4. Use a robot to patrol to the train offset marker point, collect images, remove outliers and filter the collected images, obtain the ICP registration transformation matrix from the processed file, and perform registration and stitching in combination with the ICP algorithm to obtain the target point cloud and the point cloud to be registered. S5. Obtain features from the target point cloud and the point cloud to be registered, and combine them with the NDT algorithm to obtain the NDT transformation matrix; S6. Use the NDT transformation matrix as the initial matrix for ICP to register the target point cloud and the point cloud to be registered, and obtain the offset value. S7, Dynamic calibration offset value of robotic arm; Specifically, the method for obtaining the target point cloud and the point cloud to be registered in step S4 is as follows: S41. The robotic arm moves a fixed distance to take pictures, expanding the field of view of the captured images, and an initial matrix for point cloud registration is given. Record the position of the robot arm taking the picture as the center position. The end point moves horizontally left and right to collect data at the middle position. That is, the left side of the photo position And the photo location on the right Take three photos: left, center, and right. The photos are in PLY format and labeled with the given names. , and Save it in the specified path; S42. Read three PLY files and perform outlier removal and filtering on the three PLY files; S43. Obtain the transformation matrix of ICP registration and perform registration stitching in combination with the ICP algorithm; S44. Obtain the target point cloud and the point cloud to be registered; The specific steps for dynamically calibrating the offset value of the robotic arm in step S7 are as follows: S71. Use the train stopping offset value obtained in step S6 to perform pose compensation on all inspection points on the map. S72. When the robot performs inspection tasks, at each offset calibration point, it first takes a picture of the selected train component at each calibration point, puts the picture into the target detection network, and verifies the result of component target detection. S73. Combining the two constraints of the detected component probability and the label box being close to the center of the entire image in the results, calculate the pixel coordinate distance between the component and the image center, and use the proportional relationship between the pixel coordinate system and the world coordinate system. The error value is calculated, and the same operation is performed at each calibration point to calculate the average error value, update the calculated train stopping offset value, and perform the inspection task.

2. The train stop deviation detection method according to claim 1, characterized in that: The method for selecting inspection points in step S2 is as follows: Step 1: Preset the track angle θ: The angle between the track where the train stops and the horizontal direction is defined as θ. Step 2: Select image acquisition points: Based on the train structure, select robot vision image acquisition points that meet the requirements; Step 3: Adjust the sampling angle: To eliminate the offset error caused by the shooting angle, the mounting angle of the 3D camera was adjusted to be consistent with the angle θ of the vehicle track; Step 4: Record the inspection points: After image acquisition is completed, the location and its corresponding angle θ are recorded in the 3D point cloud map. This location serves as a marker for calculating train offset. It serves as a reference for subsequent offset calculations and robot navigation and positioning.

3. The train stop deviation detection method according to claim 1, characterized in that: The specific steps of updating the train offset marker position in step S3 are as follows: S31. The robot navigates to the initial marker point, collects images of the marker point, and trains the dataset. S32. Detect the image using the target algorithm to determine whether renavigation to a new marker point is required; If a marker is detected in the image, its location is recorded as the train offset marker location. ; If no marker component is detected in the image, it indicates that there is a deviation in the current navigation position of the robot. The robot position needs to be corrected, the navigation needs to be reissued, and the process jumps to step S31 until the train offset marker point is selected. S33, Correct the robot's positioning and pose.

4. The train stop deviation detection method according to claim 3, characterized in that: The specific method for training the dataset in step S31 is as follows: S311. Navigate the robot to the marker location to collect the dataset, and label the dataset with a certain part of the train as the part to be identified. S312. Use the target detection algorithm in visual recognition to train the dataset. In order to ensure that the robot can recognize the labeled parts in the pictures, the label categories are divided into three categories: the first category is the left half of the part, the second category is the whole part, and the third category is the right half of the part. S313. For the images used in the later dynamic calibration of the robotic arm, select the previous one. Each inspection point serves as a dynamic calibration offset point for the robotic arm. The robot is navigated to the calibration point, and a train component is selected as a reference standard at each calibration point. Multiple images of the component are taken, and all images are labeled to train a dataset, which will serve as a reference for subsequent dynamic calibration of the robotic arm.

5. The train stop deviation detection method according to claim 1, characterized in that, The specific method for obtaining the NDT transformation matrix in step S5 is as follows: S51. Extract the point cloud to be registered. and target point cloud The features in the data are used to generate new point clouds to be registered. and target point cloud ; S52. Register the new point cloud to be registered using the NDT algorithm. and target point cloud A transformation matrix is ​​obtained. ; S53. To accelerate convergence, the initial transformation matrix for NDT registration is obtained by traversing initial values, i.e., pre-setting... The interval size is Using the offset values, construct the transformation matrix. ,in ; this Each of the transformation matrices is used as the initial transformation matrix in the NDT algorithm for registration calculation, resulting in... The transformation matrix corresponding to the minimum registration score is the final transformation matrix for NDT registration. .

6. A train stop deviation detection system, performing the train stop deviation detection method according to any one of claims 1-5, characterized in that, It includes an image acquisition unit, an automatic patrol unit, and a central control unit; the image acquisition unit is a 3D structured light sensor; the automatic patrol unit is a train inspection robot and a SLAM navigation and positioning system; the central control unit includes: a robotic arm photo-taking control system and a 3D point cloud image generation system, and the central control unit is connected to and controls the image acquisition unit and the automatic patrol unit. The inspection robot is a wheeled robot with a six-degree-of-freedom robotic arm mounted on its body. The end of the robotic arm is equipped with a 3D structured light sensor. The wheeled robot stops at the set point using the SLAM navigation and positioning algorithm. After it arrives at the set point, the central control unit controls the movement of the robotic arm. The 3D structured light sensor is responsible for collecting image information and generating point cloud files.

7. A computer device, comprising a memory and a processor, characterized in that, The memory stores a computer program that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 5.

Images