Intelligent vision aided obstacle avoidance and path planning system for roadheader
By combining visual sensors and lidar with multi-source sensor data, intelligent vision-assisted obstacle avoidance and path planning of tunneling machines are achieved, solving the problems of low intelligence and inflexible path planning in traditional methods, and improving the operating efficiency and safety of tunneling machines.
Patent Information
- Application Number
- CN202610876785.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-17
- Publication Date
- 2026-07-14
AI Technical Summary
Traditional tunneling machine path planning and obstacle recognition methods have low intelligence levels and cannot process obstacle information in complex environments in real time, resulting in misjudgments, omissions, and inflexible path planning, which affects operational efficiency and safety.
The system uses visual sensors and lidar to collect environmental data of the target area of the tunneling machine. Combined with obstacle recognition, attitude acquisition, coordinate transformation, model building and path planning modules, it generates the optimal path for the tunneling machine, enabling real-time data updates and dynamic path adjustments.
It improves the intelligence and path recognition capabilities of tunneling machines, enabling them to flexibly respond to environmental changes and improve operational efficiency and safety.
Smart Images
Figure CN122384840A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of path planning technology, and in particular to an intelligent vision-assisted obstacle avoidance and path planning system for tunneling machines. Background Technology
[0002] Tunnel boring machines (TBMs) are key pieces of equipment in modern tunnel construction, widely used in urban underground construction, large tunnels, high-speed railways, subways, and other projects. With the increasing scale of projects and the complexity of construction environments, traditional TBM path planning and obstacle recognition methods face many challenges.
[0003] Traditional tunnel boring machines (TBMs) rely heavily on manual operation and simple sensing systems for path planning and obstacle recognition. For example, some traditional methods use only robotic arms or simple sensing devices (such as a single radar or camera) to perform basic environmental detection and manually adjust to avoid obstacles. These technologies have low levels of intelligence and cannot process constantly changing obstacle information in complex environments in real time.
[0004] Furthermore, traditional tunneling machine path planning and obstacle recognition methods have very limited ability to identify obstacles with complex shapes or significant obstructions (such as pipes and protruding rocks), easily leading to misjudgments or missed detections. In addition, traditional path planning methods typically use preset static paths, which, due to a lack of real-time data updates and flexible path adjustments, cannot flexibly respond to new obstacles in the environment or dynamic changes in the tunneling machine, thus failing to effectively improve the operating efficiency and safety of the tunneling machine. Summary of the Invention
[0005] To address the aforementioned technical problems, this invention provides an intelligent vision-assisted obstacle avoidance and path planning system for tunneling machines. The technical solution of this invention is as follows: The intelligent vision-assisted obstacle avoidance and path planning system for tunneling machines includes: The data acquisition module is used to collect raw environmental data of the target area of the tunneling machine at each time within a preset time period before the current time through visual sensors and lidar, and to generate ground data of the target area at each time in the ground coordinate system based on the raw environmental data at each time. The obstacle recognition module is used to identify all obstacles and their information in the raw environmental data at each moment. The attitude acquisition module is used to acquire the attitude parameters of the tunneling machine at each moment within a preset time period before the current moment; The coordinate transformation module is used to map the ground data at each moment to the tunneling machine's coordinate system based on the tunneling machine's attitude parameters at each moment, thereby generating tunneling point cloud data at each moment. The modeling module is used to calculate the state estimation parameters of the tunnel boring machine at each moment based on the attitude parameters of the tunnel boring machine at each moment, and to create a 3D environmental model of the target area at the current moment based on the state estimation parameters of the tunnel boring machine at each moment and the LOD algorithm. The path planning module is used to plan the path based on all obstacles and their information and the 3D environmental model at the current moment, and generate the optimal path for the tunneling machine.
[0006] Preferably, the data acquisition module includes: The visual perception acquisition unit is used to acquire the original image data of the target area of the tunneling machine at each moment within a preset time period before the current moment through the visual sensor configured on the tunneling machine, and to acquire the original point cloud data of the target area of the tunneling machine at each moment within a preset time period before the current moment through the lidar configured on the tunneling machine. The original environmental data at each moment is composed of the original image data and the original point cloud data. The preprocessing unit is used to preprocess and standardize the raw environmental data at each time step to obtain standard image data and standard point cloud data at each time step. The ground coordinate system mapping unit maps the standard image data and standard point cloud data at each moment to the ground coordinate system, generating ground data of the target area at each moment in the ground coordinate system.
[0007] Preferably, the obstacle recognition module includes: The visual obstacle recognition unit is used to label all visual obstacles and their obstacle information in the standard image data at each time step according to the pre-trained obstacle recognition neural network, forming the first obstacle set at each time step; The radar obstacle recognition unit is used to label all lidar obstacles and their obstacle information in the standard point cloud data at each time step according to the point cloud segmentation algorithm, forming a second obstacle set at each time step; The obstacle information recognition unit is used to obtain the union of the first obstacle set and the second obstacle set at each time moment, so as to obtain all obstacles and their obstacle information in the original environmental data at each time moment.
[0008] Preferably, the coordinate transformation module includes: The tunneling machine coordinate system construction unit is used to construct the tunneling machine coordinate system with the geometric center of the tunneling machine body as the origin, the tunneling direction of the tunneling machine as the x-axis, the right side of the tunneling direction of the tunneling machine as the y-axis, and the z-axis forming a right-handed coordinate system with the x-axis and y-axis. The transformation matrix construction unit is used to construct the transformation matrix between the ground coordinate system and the tunneling machine coordinate system at each moment; The transformation unit is used to transform all ground coordinate points in the ground data at each time moment into the tunneling machine coordinate system using a nonlinear projection algorithm based on the transformation matrix at each time moment. This results in the tunneling machine coordinate points in the tunneling machine coordinate system for all ground coordinate points at each time moment, and the tunneling point cloud data for each time moment is composed of all tunneling machine coordinate points at each time moment.
[0009] Preferably, the modeling module includes: The first 3D model building unit is used to generate the first 3D model of the target area at the initial moment based on the tunneling point cloud data and all obstacles and their information, combined with 3D modeling technology. The dynamic update unit is used to calculate the state estimation parameters of the tunneling machine at each moment based on the attitude parameters of the tunneling machine at each moment, and iteratively update the first three-dimensional model at the initial moment based on the state estimation parameters at each moment and all obstacles and their information, and generate the second three-dimensional model of the target area at the current moment. The region division unit is used to divide the target region into multiple initial regions. Based on the clustering algorithm, the multiple initial regions are clustered by feature to obtain multiple unit regions. The unit region model of each unit region is generated according to the second three-dimensional model at the current time. The rendering hierarchy generation unit is used to calculate the distance from the tunneling machine to each unit area based on the current state estimation parameters of the tunneling machine, and generate the rendering hierarchy between the tunneling machine and each unit area according to the distance threshold-rendering hierarchy mapping table; The dynamic rendering unit is used to dynamically adjust the rendering detail level of each unit area modeling according to the LOD algorithm and the rendering layer between the tunneling machine and each unit area, and generate the current three-dimensional environmental model of the target area. The environmental three-dimensional model includes tunneling machine data, obstacles and obstacle information.
[0010] Preferably, the dynamic update unit updates according to the tunneling machine. k Attitude parameters of the tunnel boring machine at any given time are calculated. k When estimating the state parameters at time t, it is achieved through formula (2): , (2); In formula (2), Indicates tunneling machine k State estimation parameters at time 10:00 Indicates tunneling machine k- State estimation parameters at time 1 Represents the state transition function. express k Systematic error at time, expressk The state covariance matrix at time t, express k- The state covariance matrix at time 1, express k Time-based process noise, express k The state transition matrix at time t, Indicates tunneling machine k Attitude parameters at any given time.
[0011] Preferably, the path planning module includes: The obstacle avoidance priority calculation unit is used to calculate the distance from each obstacle to the tunneling machine in the current environmental 3D model of the target area based on the tunneling machine data in the current environmental 3D model, and to calculate the obstacle avoidance priority of the tunneling machine for each obstacle at the current time based on the distance from each obstacle to the tunneling machine in the current environmental 3D model and each obstacle and its information. The path planning unit is used to generate an initial path from the current position of the tunnel boring machine to the target endpoint based on the Dixtra algorithm, combined with the current position of the tunnel boring machine and the obstacle avoidance priority for each obstacle. The current position of the tunnel boring machine is determined by the attitude parameters of the tunnel boring machine at the current moment. The dynamic path adjustment unit is used to split the initial path of the tunneling machine into multiple path nodes, path endpoints, and path start points ordered according to the tunneling direction; calculate the minimum cost of each path node; and connect multiple connectable path nodes, path endpoints, and path start points with the minimum sum of minimum costs according to the tunneling direction of the tunneling machine to generate the optimal path.
[0012] Preferably, the target obstacle is defined as any obstacle in the 3D environmental model. The obstacle avoidance priority calculation unit calculates the obstacle avoidance priority of the tunnel boring machine for the target obstacle at the current moment based on the distance from the target obstacle to the tunnel boring machine and the obstacle information of the target obstacle in the 3D environmental model at the current moment. When, this is achieved through formula (3): (3); In formula (3), This represents the total weighting coefficient. This represents the volume weighting factor, and this represents the influence range weighting factor. This represents the distance from the target obstacle to the tunneling machine in the current 3D environmental model. V Indicates the volume of the target obstacle. Indicates the preset range of influence of the target obstacle.
[0013] Preferably, the dynamic path adjustment unit calculates the path nodes. minimum cost When, this is achieved through formula (4): (4); In formula (4), This represents the function that takes the minimum value. Indicates that the tunneling machine has reached the path node. The minimum cost, Indicates that the tunneling machine has reached the path node. The minimum cost, Indicates the tunneling machine starts from the path node. To path node The cost, Indicates the tunneling machine starts from the path node. To path node The cost.
[0014] All of the above-mentioned optional technical solutions can be combined arbitrarily, and the present invention will not provide a detailed description of the structure after each combination.
[0015] By means of the above solution, the beneficial effects of the present invention are as follows: By using visual sensors and lidar to collect raw environmental data of the target area of the tunneling machine at each moment, visual-assisted obstacle avoidance and path planning are carried out. This process combines environmental data collected by multiple sensors, which not only makes the process more intelligent but also makes the subsequent path planning more accurate.
[0016] By identifying all obstacles and their information in the raw environmental data at each moment, an accurate data foundation is provided for subsequent obstacle avoidance and path planning. Compared with traditional path planning methods, obstacles are fully considered during path planning, and the identification capability is significantly improved.
[0017] By creating a 3D environmental model of the target area at the current moment based on the state estimation parameters of the tunneling machine at each moment and the LOD algorithm, and generating the optimal path for the tunneling machine based on all obstacles and their information and the 3D environmental model, intelligent visual-assisted obstacle avoidance and path adjustment are achieved by updating the 3D environmental model in real time and dynamically adjusting the optimal path. This enables flexible response to new obstacles in the environment or dynamic changes in the tunneling machine, thereby effectively improving the operating efficiency and safety of the tunneling machine.
[0018] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Attached Figure Description
[0019] Figure 1This is a schematic diagram of the intelligent vision-assisted obstacle avoidance and path planning system for tunneling machines provided by the present invention. Detailed Implementation
[0020] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.
[0021] like Figure 1 As shown in the embodiment of the present invention, the intelligent vision-assisted obstacle avoidance and path planning system for tunneling machines includes: The data acquisition module is used to collect raw environmental data of the target area of the tunneling machine at each time within a preset time period before the current time through visual sensors and lidar, and to generate ground data of the target area at each time in the ground coordinate system based on the raw environmental data at each time. The obstacle recognition module is used to identify all obstacles and their information in the raw environmental data at each moment. The attitude acquisition module is used to acquire the attitude parameters of the tunneling machine at each moment within a preset time period before the current moment; The coordinate transformation module is used to map the ground data at each moment to the tunneling machine's coordinate system based on the tunneling machine's attitude parameters at each moment, thereby generating tunneling point cloud data at each moment. The modeling module is used to calculate the state estimation parameters of the tunnel boring machine at each moment based on the attitude parameters of the tunnel boring machine at each moment, and to create a 3D environmental model of the target area at the current moment based on the state estimation parameters of the tunnel boring machine at each moment and the LOD algorithm. The path planning module is used to plan the path based on all obstacles and their information and the 3D environmental model at the current moment, and generate the optimal path for the tunneling machine.
[0022] Specifically, in the data acquisition module, the visual sensor is a camera or an infrared camera. The target area is the pre-defined obstacle avoidance zone around the tunneling machine. The raw environmental data includes raw image data and raw point cloud data. The preset time period is 20 minutes in this embodiment of the invention.
[0023] In the obstacle recognition module, obstacles include rocks, equipment, or piles of debris; obstacle information includes the size, location, and preset range of influence of the obstacle.
[0024] In the attitude acquisition module, attitude parameters include the position, attitude angle, and speed of the tunneling machine. The attitude parameters of the tunneling machine at each moment can be obtained by sensors such as positioning sensors, angle sensors, and speed sensors pre-installed on the tunneling machine.
[0025] In the coordinate transformation module, the ground data at each moment is mapped to the tunneling machine's coordinate system based on the tunneling machine's attitude parameters at each moment. This ensures that the ground data is synchronized with the actual position of the tunneling machine, providing an accurate data foundation for the subsequent construction of the environmental 3D model.
[0026] In the modeling module, the state estimation parameters include pose parameters and pose correction parameters that affect the state of the tunnel boring machine.
[0027] In the path planning module, the optimal path of the tunnel boring machine is continuously adjusted dynamically based on real-time obstacle information and 3D environmental model. This ensures the continuity and safety of the tunnel boring machine's operation. Furthermore, different optimal paths can be selected according to changes in the underground environment, allowing the tunnel boring machine to flexibly cope with complex underground working environments.
[0028] In one specific embodiment, the data acquisition module includes: The visual perception acquisition unit is used to acquire the original image data of the target area of the tunneling machine at each moment within a preset time period before the current moment through the visual sensor configured on the tunneling machine, and to acquire the original point cloud data of the target area of the tunneling machine at each moment within a preset time period before the current moment through the lidar configured on the tunneling machine. The original environmental data at each moment is composed of the original image data and the original point cloud data. The preprocessing unit is used to preprocess and standardize the raw environmental data at each time step to obtain standard image data and standard point cloud data at each time step. The ground coordinate system mapping unit maps the standard image data and standard point cloud data at each moment to the ground coordinate system, generating ground data of the target area at each moment in the ground coordinate system.
[0029] Specifically, in the visual perception acquisition unit, the acquisition frequency configuration of the visual sensor (such as a camera) and the lidar is consistent, thereby ensuring the temporal consistency between the original image data and the original point cloud data.
[0030] In the preprocessing unit, through preprocessing and standardization, noise data in the raw environmental data can be removed while eliminating sensor dimension differences, so as to provide accurate ground data for subsequent analysis.
[0031] In the ground coordinate system mapping unit, the mapping is based on the relative positional relationship between the ground coordinate system and the tunneling machine. The ground coordinate system takes a preset ground reference point in the target area as its origin, the tunneling direction of the tunneling machine as the x-axis, the direction perpendicular to the ground upwards as the z-axis, and the direction perpendicular to both the x-axis and z-axis as the y-axis. The mapping of standard image data and standard point cloud data at each moment to the ground coordinate system is achieved through a preset pose relationship between the visual sensor and the LiDAR and the ground coordinate system.
[0032] In one specific embodiment, the obstacle recognition module includes: The visual obstacle recognition unit is used to label all visual obstacles and their obstacle information in the standard image data at each time step according to the pre-trained obstacle recognition neural network, forming the first obstacle set at each time step; The radar obstacle recognition unit is used to label all lidar obstacles and their obstacle information in the standard point cloud data at each time step according to the point cloud segmentation algorithm, forming a second obstacle set at each time step; The obstacle information recognition unit is used to obtain the union of the first obstacle set and the second obstacle set at each time moment, so as to obtain all obstacles and their obstacle information in the original environmental data at each time moment.
[0033] Specifically, in the visual obstacle recognition unit, visual obstacles refer to obstacles identified by the visual sensor. The pre-trained obstacle recognition neural network uses a convolutional neural network (CNN), which is trained on a large amount of image data labeled with obstacles and obstacle information.
[0034] In the radar obstacle recognition unit, lidar obstacles refer to obstacles identified by lidar. The point cloud segmentation algorithm employs the Random Sample Consensus Algorithm (RANSAC).
[0035] In the obstacle information recognition unit, when obtaining the union of the first obstacle set and the second obstacle set at each time moment, assuming the first obstacle set is {A,B,C} and the second obstacle set is {A,B,C,D}, then the set of all obstacles and their obstacle information is {A,B,C,D}, where A, B, C and D represent different obstacles and their obstacle information.
[0036] In one specific embodiment, the coordinate transformation module includes: The tunneling machine coordinate system construction unit is used to construct the tunneling machine coordinate system with the geometric center of the tunneling machine body as the origin, the tunneling direction of the tunneling machine as the x-axis, the right side of the tunneling direction of the tunneling machine as the y-axis, and the z-axis forming a right-handed coordinate system with the x-axis and y-axis. The transformation matrix construction unit is used to construct the transformation matrix between the ground coordinate system and the tunneling machine coordinate system at each moment; The transformation unit is used to transform all ground coordinate points in the ground data at each time moment into the tunneling machine coordinate system using a nonlinear projection algorithm based on the transformation matrix at each time moment. This results in the tunneling machine coordinate points in the tunneling machine coordinate system for all ground coordinate points at each time moment, and the tunneling point cloud data for each time moment is composed of all tunneling machine coordinate points at each time moment.
[0037] Specifically, in the transformation unit, for a ground coordinate point at a certain moment... A nonlinear projection algorithm is used to transform it into the tunneling machine coordinate system at that moment, thus obtaining the corresponding tunneling machine coordinate points. The calculation formula is: ; This represents the transformation matrix between the ground coordinate system and the tunneling machine coordinate system at that moment. The transformation matrix at that moment includes the rotation matrix and translation vector from the ground coordinate system to the tunneling machine coordinate system at that moment, and the rotation matrix and translation vector are determined by the attitude parameters of the tunneling machine at that moment.
[0038] In one specific embodiment, the transformation matrix construction unit constructs the transformation matrix between the ground coordinate system and the tunneling machine coordinate system at a certain moment. When, it is achieved through formula (1): (1); In formula (1), This represents the element in the first row and first column of the rotation matrix. This represents the element in the first row and second column of the rotation matrix. This represents the element in the 1st row and 3rd column of the rotation matrix. This represents the element in the 2nd row and 1st column of the rotation matrix. This represents the element in the 2nd row and 2nd column of the rotation matrix. This represents the element in the 2nd row and 3rd column of the rotation matrix. This represents the element in the 3rd row and 1st column of the rotation matrix. This represents the element in the 3rd row and 2nd column of the rotation matrix. This represents the element in the 3rd row and 3rd column of the rotation matrix. This represents the first element of the translation vector from the origin of the ground coordinate system to the origin of the tunneling machine coordinate system. This represents the second element in the translation vector. This represents the third element in the translation vector.
[0039] In one specific embodiment, the modeling module includes: The first 3D model building unit is used to generate the first 3D model of the target area at the initial moment based on the tunneling point cloud data and all obstacles and their information, combined with 3D modeling technology. The dynamic update unit is used to calculate the state estimation parameters of the tunneling machine at each moment based on the attitude parameters of the tunneling machine at each moment, and iteratively update the first three-dimensional model at the initial moment based on the state estimation parameters at each moment and all obstacles and their information, and generate the second three-dimensional model of the target area at the current moment. The region division unit is used to divide the target region into multiple initial regions. Based on the clustering algorithm, the multiple initial regions are clustered by feature to obtain multiple unit regions. The unit region model of each unit region is generated according to the second three-dimensional model at the current time. The rendering hierarchy generation unit is used to calculate the distance from the tunneling machine to each unit area based on the current state estimation parameters of the tunneling machine, and generate the rendering hierarchy between the tunneling machine and each unit area according to the distance threshold-rendering hierarchy mapping table; The dynamic rendering unit is used to dynamically adjust the rendering detail level of each unit area modeling according to the LOD algorithm and the rendering layer between the tunneling machine and each unit area, and generate the current three-dimensional environmental model of the target area. The environmental three-dimensional model includes tunneling machine data, obstacles and obstacle information.
[0040] Specifically, in the first 3D model building unit, the tunneling point cloud data at the initial moment refers to the tunneling point cloud data at the first moment within the preset time period; the 3D modeling technology adopts real-time localization and mapping (SLAM) technology.
[0041] In the dynamic update unit, when iteratively updating the first three-dimensional model of the target region at the initial time, the state estimation parameters of the next time step are first calculated. The first three-dimensional model at the initial time step is updated based on the state estimation parameters of the next time step to obtain the first three-dimensional model at the next time step. This process is repeated iteratively to obtain the second three-dimensional model at the current time step.
[0042] In the region division unit, the initial region includes region types such as obstacle regions and passage regions. When performing feature clustering on multiple initial regions based on a clustering algorithm, the edges of each initial region are extracted, and multiple initial regions of the same region type with connected edges are clustered into a single unit region. Based on the region division of each unit region, the corresponding unit region model is constructed in the second 3D model.
[0043] In the rendering level generation unit, the distance from the tunnel boring machine to a certain unit area is calculated based on the pose parameters in the current state estimation parameters of the tunnel boring machine. If the distance is greater than a first distance threshold, the rendering level is 1; if the distance is less than a second distance threshold, the rendering level is 3; and if the distance is between the first and second distance thresholds, the rendering level is 2. The first distance threshold is greater than the second distance threshold. The distance threshold-rendering level mapping table includes the correspondence between distance thresholds and rendering levels.
[0044] In the dynamic rendering unit, the Level of Detail (LOD) algorithm is used to dynamically adjust the rendering detail level of each unit region's modeling. Specifically, when the tunneling machine is far from a certain unit region, the unit region modeling uses a simplified model with a low polygon count, resulting in a lower rendering detail level; when the tunneling machine is close to a certain unit region, the unit region modeling uses a refined model with a high polygon count, resulting in a higher rendering detail level, to reduce unnecessary rendering burden. Assume the distance between the tunneling machine and a certain unit region is... ,according to After determining the rendering level of detail (LTD), the LOD algorithm selects different LLD levels based on the rendering hierarchy and uses a piecewise function to determine when to switch between different LLD levels. The LOD algorithm formula is as follows: ; ; ; These represent different levels of rendering detail. and These are the first and second distance thresholds set, respectively. When the distance exceeds the first distance threshold (i.e., the rendering level is 1), the unit region modeling adopts the rendering detail level of a low-poly model. ;when When the distance is less than the second distance threshold (i.e., the rendering level is 3), the unit region modeling uses a high polygon count rendering detail level. ,when When the distance falls between the first and second distance thresholds (i.e., the rendering level is 2), the unit region modeling for this unit region adopts the rendering level of detail. .
[0045] In one specific embodiment, the dynamic update unit updates according to the tunneling machine. k Attitude parameters of the tunnel boring machine at any given time are calculated. k When estimating the state parameters at time t, it is achieved through formula (2): , (2); In formula (2), Indicates tunneling machine k State estimation parameters at time 10:00 Indicates tunneling machine k- State estimation parameters at time 1 Represents the state transition function. express k Systematic error at time, express k The state covariance matrix at time t, express k- The state covariance matrix at time 1, express k Time-based process noise, express k The state transition matrix at time t, Indicates tunneling machine k Attitude parameters at any given time.
[0046] Specifically, when determining the state covariance matrix at each time step, an initial state covariance matrix (initial time step) is first constructed using empirical values. Then, the initial state covariance matrix is iteratively updated based on the state transition matrix and process noise of the tunneling machine at each time step, resulting in the state covariance matrix at each time step. Process noise represents the system description error and can be determined using the state estimation parameters of the tunneling machine at each time step. The state transition matrix is obtained by taking the partial derivative of the state estimation parameters in the state transition function of the tunneling machine at a certain time step. Specifically, when determining the state transition function of the tunneling machine at a certain time step, the state estimation parameters for that time step are first calculated based on the attitude parameters at that time step. Then, the pose parameters from the state estimation parameters at that time step are input into the dynamic model determined based on empirical values.
[0047] In one specific embodiment, the path planning module includes: The obstacle avoidance priority calculation unit is used to calculate the distance from each obstacle to the tunneling machine in the current environmental 3D model of the target area based on the tunneling machine data in the current environmental 3D model, and to calculate the obstacle avoidance priority of the tunneling machine for each obstacle at the current time based on the distance from each obstacle to the tunneling machine in the current environmental 3D model and each obstacle and its information. The path planning unit is used to generate an initial path from the current position of the tunnel boring machine to the target endpoint based on the Dixtra algorithm, combined with the current position of the tunnel boring machine and the obstacle avoidance priority for each obstacle. The current position of the tunnel boring machine is determined by the attitude parameters of the tunnel boring machine at the current moment. The dynamic path adjustment unit is used to split the initial path of the tunneling machine into multiple path nodes, path endpoints, and path start points ordered according to the tunneling direction; calculate the minimum cost of each path node; and connect multiple connectable path nodes, path endpoints, and path start points with the minimum sum of minimum costs according to the tunneling direction of the tunneling machine to generate the optimal path.
[0048] Specifically, in the obstacle avoidance priority calculation unit, the tunneling machine data in the 3D environmental model includes the tunneling machine's position and attitude data. When calculating the distance between a certain obstacle and the tunneling machine in the 3D environmental model, it is done using Euclidean algorithm based on the position data.
[0049] In the path planning unit, this embodiment of the invention employs the Dijkstra algorithm when generating the initial path. Specifically, the current position of the tunneling machine, the obstacle avoidance priority of the tunneling machine for each obstacle, and the target destination are input into the Dijkstra algorithm, which automatically generates the initial path. Path planning using the Dijkstra algorithm ensures that the tunneling machine can avoid obstacles, guaranteeing both its working efficiency and safety.
[0050] In a further embodiment of the present invention, the path planning module further includes a tunneling machine adjustment unit, used to adjust the tunneling machine's speed and direction based on the optimal path, the tunneling machine's current data (including travel speed and direction), and a preset response mechanism. Specifically, the preset response mechanism stores the correspondence between the tunneling machine's path, travel speed, and direction and the tunneling machine's travel speed and direction at the next moment. Therefore, after obtaining the optimal path and the tunneling machine's current data, they are compared with the preset response mechanism to obtain the tunneling machine's travel speed and direction at the next moment.
[0051] In the tunnel boring machine (TBM) adjustment unit, when a new obstacle appears or a known obstacle changes during the TBM's movement, the machine's speed and direction are dynamically adjusted to avoid collisions and ensure the continuity and safety of the TBM's operation. Through real-time adjustments, it can flexibly respond to complex underground working environments, improving the safety and efficiency of obstacle avoidance for the TBM.
[0052] In one specific embodiment, the target obstacle is defined as any obstacle in the 3D environmental model. The obstacle avoidance priority calculation unit calculates the obstacle avoidance priority of the tunnel boring machine for the target obstacle at the current moment based on the distance from the target obstacle to the tunnel boring machine in the 3D environmental model at the current moment and the obstacle information of the target obstacle. When, this is achieved through formula (3): (3); In formula (3), This represents the total weighting coefficient. This represents the volume weighting factor. This represents the weighting coefficient for the scope of influence. This represents the distance from the target obstacle to the tunneling machine in the current 3D environmental model. V Indicates the volume of the target obstacle. Indicates the preset range of influence of the target obstacle.
[0053] In one specific embodiment, the dynamic path adjustment unit calculates path nodes. minimum cost When, this is achieved through formula (4): (4); In formula (4), This represents the function that takes the minimum value. Indicates that the tunneling machine has reached the path node. The minimum cost, Indicates that the tunneling machine has reached the path node. The minimum cost, Indicates the tunneling machine starts from the path node. To path node The cost, Indicates the tunneling machine starts from the path node. To path node The cost.
[0054] Specifically, in this embodiment of the invention, the cost of the tunneling machine traveling from path node 1 to path node 2 is represented by the distance between path node 1 and path node 2.
[0055] In summary, advanced technologies such as nonlinear projection algorithms, 3D modeling techniques, and dynamic programming have enabled the tunneling machine to achieve precise positioning, path planning, and obstacle avoidance in dynamic underground environments. The system provided in this embodiment also features real-time performance monitoring and optimization capabilities, dynamically adjusting the rendering detail level of the 3D environmental model based on the tunneling machine's travel distance to ensure efficient operation even under high load conditions.
[0056] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A tunneling machine intelligent vision-assisted obstacle avoidance and path planning system, characterized in that, include: The data acquisition module is used to collect raw environmental data of the target area of the tunneling machine at each time within a preset time period before the current time through visual sensors and lidar, and to generate ground data of the target area at each time in the ground coordinate system based on the raw environmental data at each time. The obstacle recognition module is used to identify all obstacles and their information in the raw environmental data at each moment. The attitude acquisition module is used to acquire the attitude parameters of the tunneling machine at each moment within a preset time period before the current moment; The coordinate transformation module is used to map the ground data at each moment to the tunneling machine's coordinate system based on the tunneling machine's attitude parameters at each moment, thereby generating tunneling point cloud data at each moment. The modeling module is used to calculate the state estimation parameters of the tunnel boring machine at each moment based on the attitude parameters of the tunnel boring machine at each moment, and to create a 3D environmental model of the target area at the current moment based on the state estimation parameters of the tunnel boring machine at each moment and the LOD algorithm. The path planning module is used to plan the path based on all obstacles and their information and the 3D environmental model at the current moment, and generate the optimal path for the tunneling machine.
2. The intelligent vision-assisted obstacle avoidance and path planning system for tunneling machines according to claim 1, characterized in that, The data acquisition module includes: The visual perception acquisition unit is used to acquire the original image data of the target area of the tunneling machine at each moment within a preset time period before the current moment through the visual sensor configured on the tunneling machine, and to acquire the original point cloud data of the target area of the tunneling machine at each moment within a preset time period before the current moment through the lidar configured on the tunneling machine. The original environmental data at each moment is composed of the original image data and the original point cloud data. The preprocessing unit is used to preprocess and standardize the raw environmental data at each time step to obtain standard image data and standard point cloud data at each time step. The ground coordinate system mapping unit maps the standard image data and standard point cloud data at each moment to the ground coordinate system, generating ground data of the target area at each moment in the ground coordinate system.
3. The intelligent vision-assisted obstacle avoidance and path planning system for tunneling machines according to claim 2, characterized in that, The obstacle recognition module includes: The visual obstacle recognition unit is used to label all visual obstacles and their obstacle information in the standard image data at each time step according to the pre-trained obstacle recognition neural network, forming the first obstacle set at each time step; The radar obstacle recognition unit is used to label all lidar obstacles and their obstacle information in the standard point cloud data at each time step according to the point cloud segmentation algorithm, forming a second obstacle set at each time step; The obstacle information recognition unit is used to obtain the union of the first obstacle set and the second obstacle set at each time moment, so as to obtain all obstacles and their obstacle information in the original environmental data at each time moment.
4. The intelligent vision-assisted obstacle avoidance and path planning system for tunneling machines according to claim 1, characterized in that, The coordinate transformation module includes: The tunneling machine coordinate system construction unit is used to construct the tunneling machine coordinate system with the geometric center of the tunneling machine body as the origin, the tunneling direction of the tunneling machine as the x-axis, the right side of the tunneling direction of the tunneling machine as the y-axis, and the z-axis forming a right-handed coordinate system with the x-axis and y-axis. The transformation matrix construction unit is used to construct the transformation matrix between the ground coordinate system and the tunneling machine coordinate system at each moment; The transformation unit is used to transform all ground coordinate points in the ground data at each time moment into the tunneling machine coordinate system using a nonlinear projection algorithm based on the transformation matrix at each time moment. This results in the tunneling machine coordinate points in the tunneling machine coordinate system for all ground coordinate points at each time moment, and the tunneling point cloud data for each time moment is composed of all tunneling machine coordinate points at each time moment.
5. The intelligent vision-assisted obstacle avoidance and path planning system for tunneling machines according to claim 1, characterized in that, The modeling module includes: The first 3D model building unit is used to generate the first 3D model of the target area at the initial moment based on the tunneling point cloud data and all obstacles and their information, combined with 3D modeling technology. The dynamic update unit is used to calculate the state estimation parameters of the tunneling machine at each moment based on the attitude parameters of the tunneling machine at each moment, and iteratively update the first three-dimensional model at the initial moment based on the state estimation parameters at each moment and all obstacles and their information, and generate the second three-dimensional model of the target area at the current moment. The region division unit is used to divide the target region into multiple initial regions. Based on the clustering algorithm, the multiple initial regions are clustered by feature to obtain multiple unit regions. The unit region model of each unit region is generated according to the second three-dimensional model at the current time. The rendering hierarchy generation unit is used to calculate the distance from the tunneling machine to each unit area based on the current state estimation parameters of the tunneling machine, and generate the rendering hierarchy between the tunneling machine and each unit area according to the distance threshold-rendering hierarchy mapping table; The dynamic rendering unit is used to dynamically adjust the rendering detail level of each unit area modeling according to the LOD algorithm and the rendering layer between the tunneling machine and each unit area, and generate the current three-dimensional environmental model of the target area. The environmental three-dimensional model includes tunneling machine data, obstacles and obstacle information.
6. The intelligent vision-assisted obstacle avoidance and path planning system for tunneling machines according to claim 5, characterized in that, The dynamic update unit updates according to the tunneling machine. k Attitude parameters of the tunnel boring machine at any given time are calculated. k When estimating the state parameters at time t, it is achieved through formula (2): ， （2）； In formula (2), Indicates tunneling machine k State estimation parameters at time 10:00 Indicates tunneling machine k- State estimation parameters at time 1 Represents the state transition function. express k Systematic error at time, express k The state covariance matrix at time t, express k- The state covariance matrix at time 1, express k Time-based process noise, express k The state transition matrix at time t, Indicates tunneling machine k Attitude parameters at any given time.
7. The intelligent vision-assisted obstacle avoidance and path planning system for tunneling machines according to claim 5, characterized in that, The route planning module includes: The obstacle avoidance priority calculation unit is used to calculate the distance from each obstacle to the tunneling machine in the current environmental 3D model of the target area based on the tunneling machine data in the current environmental 3D model, and to calculate the obstacle avoidance priority of the tunneling machine for each obstacle at the current time based on the distance from each obstacle to the tunneling machine in the current environmental 3D model and each obstacle and its information. The path planning unit is used to generate an initial path from the current position of the tunnel boring machine to the target endpoint based on the Dixtra algorithm, combined with the current position of the tunnel boring machine and the obstacle avoidance priority for each obstacle. The current position of the tunnel boring machine is determined by the attitude parameters of the tunnel boring machine at the current moment. The dynamic path adjustment unit is used to split the initial path of the tunneling machine into multiple path nodes, path endpoints, and path start points ordered according to the tunneling direction; calculate the minimum cost of each path node; and connect multiple connectable path nodes, path endpoints, and path start points with the minimum sum of minimum costs according to the tunneling direction of the tunneling machine to generate the optimal path.
8. The intelligent vision-assisted obstacle avoidance and path planning system for tunneling machines according to claim 7, characterized in that, The target obstacle is defined as any obstacle in the 3D environmental model. The obstacle avoidance priority calculation unit calculates the obstacle avoidance priority of the tunnel boring machine for the target obstacle at the current moment based on the distance from the target obstacle to the tunnel boring machine and the obstacle information of the target obstacle in the 3D environmental model at the current moment. When, this is achieved through formula (3): （3）； In formula (3), This represents the total weighting coefficient. This represents the volume weighting coefficient. This represents the weighting coefficient for the scope of influence. This represents the distance from the target obstacle to the tunneling machine in the current 3D environmental model. V Indicates the volume of the target obstacle. Indicates the preset range of influence of the target obstacle.
9. The intelligent vision-assisted obstacle avoidance and path planning system for tunneling machines according to claim 7, characterized in that, The dynamic path adjustment unit calculates path nodes. minimum cost When, this is achieved through formula (4): （4）； In formula (4), This represents the function that takes the minimum value. Indicates that the tunneling machine has reached the path node. The minimum cost, Indicates that the tunneling machine has reached the path node. The minimum cost, Indicates the tunneling machine starts from the path node. To path node The cost, Indicates the tunneling machine starts from the path node. To path node The cost.