A subway train bottom environment modeling block obstacle avoidance method and system

Abstract

The application discloses a metro train bottom environment modeling block obstacle avoidance method and system, and the method comprises the following steps: a bottom detection robot is used to take pictures in multiple directions of the surrounding environment at the bottom of the train to obtain point cloud data, the point cloud data is subjected to coordinate transformation and splicing processing to generate modeling data; the modeling data is processed to generate a projection grid map; the projection grid map is subjected to clustering processing, and according to the classification result, the point cloud space is mapped back to calculate a bounding box; and a mechanical arm is driven to move to avoid obstacles. The application is aimed at the specific application scene of the bottom of the metro train, projects the point cloud information of the depth camera in each direction of the coordinate system, performs clustering and segmentation processing on the projection map, and then calculates the bounding box, so that the number of the bounding box is greatly reduced, the mechanical arm is driven to move to avoid obstacles, the processing speed is fast, and the requirement of the bottom inspection efficiency can be met.

Description

Technical Field

[0001] This invention relates to the field of subway train technology, specifically to a method and system for block-based obstacle avoidance by modeling the underside environment of a subway train. Background Technology

[0002] Robotic arms have always played a vital role in the industrial sector. With the advancement of industrial automation, robotic arms need to operate in more complex environments, requiring them to be able to identify and avoid obstacles. For example, a machine vision-based intelligent inspection robot for subway train undercarriages is used to inspect the undercarriage, collect images, perform visual analysis, and detect anomalies. To facilitate image capture, the robot system is typically equipped with one or two robotic arms, each with a camera mounted at its end. To prevent collisions with the undercarriage, the environment needs to be modeled. Furthermore, the intelligent inspection robot requires high inspection efficiency, typically needing to complete the inspection of an entire train within 30 minutes.

[0003] Currently, the common approach to solving obstacle avoidance problems for robotic arms is to use prior knowledge for obstacle avoidance teaching. While robots possess a certain degree of intelligence, it falls far short of expectations. One development direction is to enable robots to perceive obstacles by incorporating machine vision as an environmental perception module. Industry has begun equipping robots with 3D sensors to develop obstacle avoidance methods for robotic arms based on point cloud information. LiDAR and depth cameras are the mainstream devices for acquiring point clouds.

[0004] Currently, the mainstream method in the industry for processing point cloud data is to construct an OctoMap. OctoMap is an efficient probabilistic 3D mapping framework based on octrees, implementing a 3D mesh occupancy mapping method. An octree is a tree-like data structure used to describe 3D space, with the entire space as the root node, continuously divided. Each parent node has 8 child nodes, representing 8 smaller spaces, and then the child nodes are further divided. If the set maximum recursion depth is not reached, the division continues until the child nodes meet the resolution size or the set maximum recursion depth is reached, at which point the process stops. While OctoMap provides sufficient precision, the large number of child nodes results in slow computation speed, which does not meet the requirements for inspection efficiency. Summary of the Invention

[0005] Technical Objective: To address the aforementioned technical problems, this invention proposes a method and system for block-based obstacle avoidance in subway train bottom environment modeling. It employs a depth camera to model the environment of the subway train bottom and uses an algorithm to segment the point cloud data of the bottom model, generating bounding boxes to drive a robotic arm for obstacle avoidance. This method offers fast processing speed and meets the efficiency requirements for undercarriage inspection.

[0006] Technical solution: To achieve the above technical objectives, the present invention adopts the following technical solution:

[0007] A method for block-based obstacle avoidance in modeling the underside environment of a subway train, characterized by the following steps:

[0008] A vehicle-mounted under-vehicle detection robot takes multi-directional photos of the surrounding environment from under the vehicle to obtain point cloud data. The point cloud data is then transformed and stitched together to obtain modeling data.

[0009] The modeling data is divided into blocks to generate bounding boxes;

[0010] Based on the generated bounding box, the robotic arm of the undercarriage detection robot is driven to move and avoid obstacles.

[0011] The following steps are performed to divide the model into blocks:

[0012] Point cloud data is projected in different directions to generate a projection grid map at a fixed resolution;

[0013] The k-means algorithm is used to cluster and classify the projected raster images. The number of k-means classes is determined by the interval between the maximum and minimum depth values ​​in each projected raster image.

[0014] Based on the classification results, a connected region is calculated for each category output by the kmeans algorithm, and the bounding box is calculated back to the 3D point cloud space. That is, the point cloud data is projected, and it is determined whether each point in the point cloud data falls within the connected region after projection. If it does, the corresponding point is considered to belong to this connected region; otherwise, the corresponding point is considered not to belong to this connected region. The 3D point cloud set corresponding to each connected region is calculated, and the bounding box is generated based on the 3D point cloud set.

[0015] Calculate the density of the bounding box. The density is the number of points in the point cloud data corresponding to the bounding box divided by the volume of the bounding box. If the density of the merged bounding box is greater than the individual densities of the two bounding boxes, then the two bounding boxes can be merged.

[0016] Preferably, the following steps are performed to obtain modeling data:

[0017] Drive the robotic arm to a preset pose and record the pose information of each joint of the robotic arm;

[0018] The depth camera mounted on the end joint of the robotic arm takes pictures. The depth camera generates point cloud data based on the obtained environmental images. Combined with the current pose information of the robotic arm, the point cloud data is converted into the world coordinate system.

[0019] The pose of the end joint of the robotic arm was adjusted multiple times, and photos were taken repeatedly. The point cloud data obtained each time was converted to the world coordinate system by combining the current pose information of the robotic arm.

[0020] Combine all point cloud data transformed to the world coordinate system.

[0021] As a preferred method, the point cloud data generated by the depth camera is converted to the world coordinate system. The conversion formula is as follows:

[0022]

[0023] Where I is the identity matrix, R bc p represents the attitude of the end effector flange of the robotic arm in the world coordinate system. bc p represents the position of the end effector flange of the robotic arm in the world coordinate system. co g represents the point cloud data generated by the depth camera. bo This represents point cloud data transformed into the world coordinate system.

[0024] A block-based obstacle avoidance system for modeling the underside environment of a subway train, used to implement the method, characterized in that it includes:

[0025] The undercarriage detection robot is used to take multi-directional photos of the surrounding environment under the vehicle to obtain three-dimensional point cloud data, and then perform coordinate transformation and stitching processing on the point cloud data.

[0026] The processing device is used to project the modeling data, generate a projected raster map, perform clustering processing on the projected raster map, map it back to the point cloud space based on the classification results, and calculate and generate bounding boxes.

[0027] A drive unit is used to drive the robotic arm of the undercarriage exploration robot to move and avoid obstacles based on the generated bounding box.

[0028] An electronic device, characterized in that the electronic device includes a processor and a memory; wherein the memory is used to store a computer program, the computer program being loaded and executed by the processor to implement the method.

[0029] A computer-readable storage medium is characterized in that it is used to store a computer program; wherein the computer program implements the method when executed by a processor.

[0030] Beneficial effects: Due to the adoption of the above technical solution, the present invention has the following beneficial effects:

[0031] This invention targets the specific application scenario of the bottom of subway trains. It projects the point cloud information of the depth camera in various directions of the coordinate system, performs clustering and segmentation processing on the projection map, and then calculates the bounding box. On the one hand, it does not need to use the color image information of the camera, and on the other hand, it greatly reduces the number of bounding boxes, thus improving the data processing speed. Attached Figure Description

[0032] Figure 1 Flowchart for modeling the undercarriage;

[0033] Figure 2 Flowchart for calculating bounding boxes for point cloud segmentation;

[0034] Figure 3 This is a schematic diagram of the segmented bounding box result;

[0035] Figure 4 This is a schematic diagram of the initial pose of the robotic arm.

[0036] Figure 5 This is a schematic diagram of the robot's world coordinate system.

[0037] Figure 6 Schematic diagram of anisotropic projection grid.

[0038] Figure 7 This is a schematic diagram of the output results of the kmeans algorithm. Detailed Implementation

[0039] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0040] Example 1

[0041] This embodiment proposes a block-based obstacle avoidance method for modeling the underside environment of subway trains, including:

[0042] Step 1: Model the surrounding environment under the vehicle.

[0043] Process Corresponding Appendix Figure 1 Specifically, it includes:

[0044] Step (1.1): Operate the robotic arm to a preset pose. In this embodiment, the joint angles of the robotic arm corresponding to this preset pose are: {-1.570693,-0.9,-2.739564,-1.0491,-1.570795,-3.141519}, as shown in the attached figure. Figure 4 As shown.

[0045] Step (1.2): Take photos using the depth camera mounted on the end effector, and then combine the point cloud data generated by the depth camera with the robot arm's pose to transform it into the world coordinate system. The transformation formula is as follows:

[0046]

[0047] Where I is the identity matrix, R bc p represents the attitude of the end effector flange of the robotic arm in the world coordinate system. bc p represents the position of the end effector flange of the robotic arm in the world coordinate system. co g represents the point cloud data generated by the depth camera. boThis represents point cloud data transformed to the world coordinate system. In this embodiment, the world coordinate system is as follows: Figure 5 As shown.

[0048] Step (1.3): Adjust the position of the robotic arm's end effector and repeat step (1.2). In this embodiment, the sixth joint of the robotic arm is adjusted every 45 degrees, and the pitch angle of the end effector joint is adjusted to 15° and 75°, and a total of 16 photos are taken.

[0049] Step (1.4): Merge all the converted point cloud data (directly combine them) to generate modeling data.

[0050] Step 2: Divide the merged point cloud data into blocks. (See attached flowchart.) Figure 2 .

[0051] Step (2.1): Project the 3D point cloud in different directions to generate a raster image with a fixed resolution in any direction. The pixel value of each raster image is determined by the different directions, selecting the maximum or minimum depth value to represent it. In this embodiment, based on the actual scenario of the undercarriage detection robot's operation, the positive z-axis, positive y-axis, and negative y-axis directions of the world coordinate system are selected for projection. The resolution is set to 0.01m. Taking the positive z-axis direction as an example, the x-range is -1.5m to 1.5m, and the y-range is -1.0m to 1.0m, projecting it into a 300x200 raster image. The maximum point cloud depth in each raster is selected in the positive z-axis and positive y-axis directions, and the minimum point cloud depth in each raster is selected in the negative y-axis direction for projection. The projected raster image is as follows. Figure 6 As shown.

[0052] Step (2.2): Cluster and classify the raster grayscale image generated by projection.

[0053] In this embodiment, the k-means algorithm is used for clustering and classification. The number of k-means clusters is determined by the interval between the maximum and minimum depth values ​​in the raster image. The input to the k-means algorithm is a three-dimensional vector of x, y, and z. The output of the k-means algorithm is as follows: Figure 7 As shown.

[0054] Step (2.3): Calculate connected regions for the classification results and map them back to the 3D point cloud bounding boxes. In this embodiment, connected regions are calculated for each classification output by the kmenes algorithm, and each classification may correspond to multiple connected regions. The point cloud is projected in the manner of step (2.1). If the projected point falls within a connected region, then the point in the point cloud is considered to belong to this connected region. Calculate the 3D point cloud set corresponding to each connected region. In this embodiment, AABB bounding boxes are used to calculate the bounding boxes. Other candidate methods include OBB bounding boxes.

[0055] Step (2.4): Merge the bounding boxes. In this embodiment, calculate the density of the bounding boxes. The density is the number of points in the point cloud corresponding to the bounding box divided by the volume of the bounding box. If the density of the merged bounding boxes is greater than the individual densities of the two bounding boxes, then the two bounding boxes are considered to be merged.

[0056] Step 3: Based on the generated bounding box, drive the robotic arm of the undercarriage detection robot to move and avoid obstacles.

[0057] The robot's motion trajectory planning uses a random sampling generation tree method. After the bounding box is generated, the bounding box represents the actual obstacle. During the random sampling process, the sampling points that collide with the bounding box are deleted. Therefore, the final generated motion trajectory naturally avoids the bounding box, thus achieving obstacle avoidance.

[0058] This invention presents an algorithm for segmenting / blocking point cloud data from vehicle undercarriage modeling. It utilizes a technical approach based on projecting the point cloud into a raster depth map according to its orientation. When projecting the point cloud into the raster depth map, the maximum or minimum depth value is selected based on the orientation to represent the value of that raster cell. The number of clusters is determined based on the depth span of the entire depth map. The algorithm used in this invention generates fewer blocks; for example, for a 3.0m * 2.0m * 2.6m space, the number of blocks is 40 to 70, and these blocks are represented by bounding boxes. This improves planning speed while ensuring obstacle avoidance by the robotic arm, thus guaranteeing the final inspection efficiency. The final generated effect is shown in the attached image. Figure 3 , attached Figure 3 It is actually a color image.

[0059] Example 2

[0060] This embodiment proposes a block-based obstacle avoidance system for modeling the underside environment of a subway train, used to implement the method described in Embodiment 1, including:

[0061] The undercarriage detection robot is used to take multi-directional photos of the surrounding environment under the vehicle to obtain three-dimensional point cloud data, and then perform coordinate transformation and stitching processing on the point cloud data.

[0062] The processing device is used to project the modeling data, generate a projected raster map, perform clustering processing on the projected raster map, map it back to the point cloud space based on the classification results, and calculate and generate bounding boxes.

[0063] A drive unit is used to drive the robotic arm of the undercarriage exploration robot to move and avoid obstacles based on the generated bounding box.

[0064] In another embodiment of the present invention, an electronic device is provided, the electronic device including a processor and a memory; wherein the memory is used to store a computer program, the computer program being loaded and executed by the processor to implement the method described in Embodiment 1.

[0065] In another embodiment of the present invention, a computer-readable storage medium is provided for storing a computer program; wherein the computer program, when executed by a processor, implements the method as described in Embodiment 1.

[0066] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for block-based obstacle avoidance in modeling the underside environment of a subway train, characterized in that, Including the following steps: A vehicle-mounted under-vehicle detection robot takes multi-directional photos of the surrounding environment from under the vehicle to obtain point cloud data. The point cloud data is then transformed and stitched together to obtain modeling data. The modeling data is divided into blocks to generate bounding boxes; Based on the generated bounding box, the robotic arm of the undercarriage detection robot is driven to move and avoid obstacles. The following steps are performed to divide the model into blocks: Point cloud data is projected in different directions to generate a projection grid map at a fixed resolution; The k-means algorithm is used to cluster and classify the projected raster images. The number of k-means classes is determined by the interval between the maximum and minimum depth values ​​in each projected raster image. Based on the classification results, a connected region is calculated for each category output by the kmeans algorithm, and the bounding box is calculated back to the 3D point cloud space. That is, the point cloud data is projected, and it is determined whether each point in the point cloud data falls within the connected region after projection. If it does, the corresponding point is considered to belong to this connected region; otherwise, the corresponding point is considered not to belong to this connected region. The 3D point cloud set corresponding to each connected region is calculated, and the bounding box is generated based on the 3D point cloud set. Calculate the density of the bounding box. The density is the number of points in the point cloud data corresponding to the bounding box divided by the volume of the bounding box. If the density of the merged bounding box is greater than the density of the two bounding boxes individually, then the two bounding boxes can be merged. Based on the actual operating scenario of the undercarriage detection robot, projection was performed in the positive z-axis, positive y-axis, and negative y-axis directions of the world coordinate system. The maximum point cloud depth in each grid cell was selected in the positive z-axis and positive y-axis directions, and the minimum point cloud depth in each grid cell was selected in the negative y-axis direction for projection.

2. The method for block-based obstacle avoidance in modeling the underside environment of a subway train according to claim 1, characterized in that, Perform the following steps to obtain modeling data: Drive the robotic arm to a preset pose and record the pose information of each joint of the robotic arm; The depth camera mounted on the end joint of the robotic arm takes pictures. The depth camera generates point cloud data based on the obtained environmental images. Combined with the current pose information of the robotic arm, the point cloud data is converted into the world coordinate system. The pose of the end joint of the robotic arm was adjusted multiple times, and photos were taken repeatedly. The point cloud data obtained each time was converted to the world coordinate system by combining the current pose information of the robotic arm. Combine all point cloud data transformed to the world coordinate system.

3. The method for block-based obstacle avoidance in modeling the underside environment of a subway train according to claim 1, characterized in that, The point cloud data generated by the depth camera is converted to the world coordinate system using the following formula: in I It is the identity matrix. R bc This represents the orientation of the robotic arm's end flange in the world coordinate system. p bc This represents the position of the end flange of the robotic arm in the world coordinate system. p co Point cloud data representing the depth camera, g bo This represents point cloud data transformed into the world coordinate system.

4. A block-based obstacle avoidance system for modeling the underside environment of a subway train, used to implement the method described in any one of claims 1-3, characterized in that, include: The undercarriage detection robot is used to take multi-directional photos of the surrounding environment under the vehicle to obtain three-dimensional point cloud data, and then perform coordinate transformation and stitching processing on the point cloud data. The processing device is used to project the modeling data, generate a projected raster map, perform clustering processing on the projected raster map, map it back to the point cloud space based on the classification results, and calculate and generate bounding boxes. A drive unit is used to drive the robotic arm of the undercarriage exploration robot to move and avoid obstacles based on the generated bounding box.

5. An electronic device, characterized in that, The electronic device includes a processor and a memory; wherein the memory is used to store a computer program, which is loaded and executed by the processor to implement the method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, Used to store a computer program; wherein the computer program, when executed by a processor, implements the method as described in any one of claims 1 to 3.

Images