Dynamic path planning method and device for tunnel construction operation mechanical arm

Abstract

The invention relates to the technical field of tunnel construction, and discloses a tunnel construction operation mechanical arm dynamic path planning method and device, and the method comprises the steps: collecting environment data in a tunnel and real-time attitude parameters of a mechanical arm in real time; real-time three-dimensional grid map construction is carried out on the inner wall of the tunnel based on the environment data in the tunnel, and meanwhile real-time detection labeling is carried out on obstacles in the tunnel; a mechanical arm real-time state vector is constructed, a tunnel environment state vector is constructed, and a real-time joint state vector is constructed based on the mechanical arm real-time state vector and the tunnel environment state vector; according to the real-time joint state vector, on the basis of a pre-trained in-tunnel mechanical arm motion path model, pose adjustment parameters of the mechanical arm are obtained; wherein the in-tunnel mechanical arm motion path model is obtained based on reinforcement learning model training, and the penalty functions at least comprise a path reward function, a dynamic collision penalty function and a step number penalty function. According to the invention, the path planning problem of the mechanical arm in the tunnel construction scene can be solved.

Description

Technical Field

[0001] This invention belongs to the field of tunnel construction technology, specifically relating to a dynamic path planning method and device for a tunnel construction robotic arm. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In tunnel construction, robotic arms, as key equipment, are widely used in drilling, grouting, and material handling. The rationality of their path planning directly determines operational efficiency, construction safety, and operational accuracy, and is crucial for promoting the automation and intelligent upgrading of tunnel construction. During construction, obstacles such as temporary material stockpiles, grouting machines, and transport vehicles frequently appear and dynamically change, with static and dynamic obstacles intertwined. The condition of the tunnel wall may also undergo subtle adjustments as construction progresses. These characteristics place extremely high demands on the real-time performance, adaptability, and safety of robotic arm path planning. Existing tunnel robotic arm path planning schemes are mainly based on predefined trajectories or static environment models, which cannot quickly respond to changes in the position of dynamic obstacles within the tunnel. Their obstacle avoidance safety and path adaptability are insufficient, making it difficult to meet the actual needs of dynamic tunnel construction scenarios. Furthermore, the narrow and enclosed working space in tunnel construction scenarios, along with large fluctuations in dust concentration, easily interferes with environmental perception, affecting operational accuracy and safety. Summary of the Invention

[0004] In view of this, the present invention provides a dynamic path planning method and apparatus for a robotic arm used in tunnel construction. It is used to solve the path planning problem of a robotic arm in tunnel construction scenarios.

[0005] The first aspect of the present invention provides a dynamic path planning method for a tunnel construction robotic arm, comprising the following steps: Real-time acquisition of environmental data inside the tunnel and real-time attitude parameters of the robotic arm; Based on the environmental data inside the tunnel, a real-time three-dimensional grid map is constructed on the inner wall of the tunnel. At the same time, obstacles inside the tunnel are detected in real time, and the detected obstacles are marked on the three-dimensional grid map. A real-time state vector of the robotic arm is constructed based on its real-time posture parameters, a tunnel environment state vector is constructed based on the tunnel wall and obstacles, and a real-time joint state vector is constructed based on the real-time state vector of the robotic arm and the tunnel environment state vector. Based on the real-time joint state vector, the pose adjustment parameters of the robotic arm are obtained based on the pre-trained motion path model of the robotic arm in the tunnel. The motion path model of the robotic arm in the tunnel is trained based on a reinforcement learning model, and the penalty function includes at least the path reward function, the dynamic collision penalty function, and the step penalty function.

[0006] In some embodiments, real-time detection of obstacles within the tunnel includes: Real-time acquisition of point cloud data and image information within the tunnel; identification of target point clouds that may be obstacles from the point cloud data; Based on point cloud data, the tunnel space is divided into three-dimensional grid units according to a preset size. Each grid unit is marked with three states: free, occupied, and unknown. The occupied grid is the area where obstacles may exist. Cluster analysis is performed on the grid areas marked as occupied in the map, and each target cluster is a barrier to be identified; the outline of the barrier is identified based on image information, and the authenticity of the barrier is confirmed by position matching with the barrier to be identified; The system continuously tracks initially detected obstacles and distinguishes between static and dynamic obstacles based on their location range and contour changes.

[0007] In some embodiments, the dimensions of the real-time state vector of the robotic arm include: the real-time pose of the robotic arm, the distance between the end effector and the fixed obstacle, and the distance between the end effector and the target point; the dimensions of the tunnel environment state vector include at least: obstacle position, obstacle speed, and obstacle type identifier.

[0008] In some embodiments, the training method for the motion path model of the robotic arm inside the tunnel is as follows: Using a basic state vector and combining it with the robotic arm's motion space, a reinforcement learning model is used to pre-train the robotic arm's path planning model. During the training process, the reward function used includes at least a path reward function, a collision penalty function, and a step penalty function. The basic state vector includes the robotic arm's real-time state vector and the basic environment state vector, and the basic environment state vector includes the positions of static obstacles. By employing a tunnel joint state vector and combining it with the robotic arm's motion space, the pre-trained robotic arm path planning model is iteratively fine-tuned. During the fine-tuning process, the penalty functions used include at least the path reward function, the dynamic collision penalty function, and the step count penalty function.

[0009] In some embodiments, the dimensions of the tunnel environment state vector also include: dust concentration and the set of key geometric feature points on the tunnel inner wall; during the iterative fine-tuning of the pre-trained robotic arm path planning model, the robotic arm motion space is determined based on the dust concentration and the mapping relationship between the set dust concentration and the upper limit of the robotic arm's movement speed. The penalty function used also includes a stability detection function and a narrow passage reward function.

[0010] In some embodiments, the total reward function used in the fine-tuning phase is the sum of the individual reward functions; The dynamic collision penalty function is a piecewise function. When the robotic arm collides with another object, the value is set; otherwise, the value is 0. The path reward function, step penalty function, stability detection function, and narrow channel reward function are all products of weights and reward terms; When one or more dynamic obstacles are detected, the weights of each item in the reward function are adjusted based on the obstacle priority mapping table and the coefficient of the highest priority obstacle. When the weight of the dynamic collision penalty function increases, the weights corresponding to the narrow passage reward function and the dust environment reward function are compressed first, while the weights corresponding to the path reward function and the step penalty function are retained, so that the sum of the weights meets the set value range. When the dust concentration is lower than the set low dust threshold, the weight of the stability detection function is reduced and the weights of other functions are increased, so that the sum of the weights meets the set value range.

[0011] A second aspect of the present invention provides a dynamic path planning device for a tunnel construction robotic arm, comprising: The real-time data acquisition module is configured to collect real-time environmental data inside the tunnel and the real-time attitude parameters of the robotic arm. The raster map building module is configured to build a real-time three-dimensional raster map of the tunnel wall based on the tunnel environment data, and at the same time detect obstacles in the tunnel in real time and mark the detected obstacles in the three-dimensional raster map. The state vector construction module is configured to construct a real-time state vector of the robotic arm based on the real-time attitude parameters of the robotic arm, construct a tunnel environment state vector based on the tunnel wall and obstacles, and construct a real-time joint state vector based on the real-time state vector of the robotic arm and the tunnel environment state vector. The dynamic path planning module is configured to obtain the pose adjustment parameters of the robotic arm based on the real-time joint state vector and the pre-trained motion path model of the robotic arm in the tunnel. The motion path model of the robotic arm in the tunnel is trained based on a reinforcement learning model, and the penalty function includes at least a path reward function, a dynamic collision penalty function, and a step penalty function.

[0012] A third aspect of the present invention provides an electronic device including a processor and a memory, wherein the memory stores computer instructions that, when executed by the processor, cause the electronic device to perform the method described thereon.

[0013] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method.

[0014] A fifth aspect of the present invention provides a computer program product comprising a computer program that, when executed by a processor, implements the method described herein.

[0015] One or more of the above technical solutions can construct a 3D grid map and obstacle detection, enabling real-time identification and labeling of tunnel walls and moving and static obstacles. This allows for the construction of an environmental state vector characterizing the real-time environment inside the tunnel. By combining this environmental state vector with the real-time state vector of the robotic arm, a joint state vector can be obtained. Based on a pre-trained robotic arm motion path model within the tunnel, and using path rewards, dynamic collision penalties, and step rewards, the system adapts to the motion characteristics of dynamic obstacles. This enables rapid response to changes in the position of dynamic obstacles, effectively avoiding collision risks and better aligning with dynamic tunnel construction scenarios. Attached Figure Description

[0016] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0017] Figure 1 A flowchart of the dynamic path planning method for a tunnel construction robotic arm provided in an embodiment of this application is shown. Figure 2 The diagram shows the program module architecture of the dynamic path planning device for tunnel construction robotic arms provided in this embodiment. Detailed Implementation

[0018] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0019] In the description of the embodiments of this application, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on".

[0020] As mentioned in the background section, tunnel construction scenarios are characterized by narrow and enclosed spaces with large fluctuations in dust concentration, which can easily interfere with environmental perception. Furthermore, during construction, static and dynamic obstacles such as temporary material stockpiles and grouting machines are intertwined and frequently change, and the condition of the tunnel wall may also be adjusted as the construction progresses. Existing robotic arm path planning schemes are mostly based on predefined trajectories or static environment models, which have problems such as path redundancy, delayed response to dynamic obstacles, insufficient real-time performance, and poor scene adaptability, making it difficult to balance the safety and efficiency of robotic arm operations.

[0021] Figure 1A flowchart of an example method 100 for dynamic path planning of a robotic arm for tunnel construction, provided by one or more embodiments of the present invention, is shown, including steps S101-S104. It should be understood that method 100 may also include additional actions not shown. Method 100 is described in detail below.

[0022] S101. Real-time acquisition of environmental data inside the tunnel and real-time attitude parameters of the robotic arm; S102. Construct a real-time three-dimensional grid map of the tunnel wall based on the tunnel environment data, and simultaneously detect obstacles in the tunnel in real time and mark the detected obstacles in the three-dimensional grid map. S103. Construct a real-time state vector of the robotic arm based on the real-time posture parameters of the robotic arm, construct a tunnel environment state vector based on the tunnel wall and obstacles, and construct a real-time joint state vector based on the real-time state vector of the robotic arm and the tunnel environment state vector. S104. Based on the real-time joint state vector and the pre-trained motion path model of the robotic arm in the tunnel, obtain the pose adjustment parameters of the robotic arm; wherein, the motion path model of the robotic arm in the tunnel is trained based on a reinforcement learning model, and the penalty function includes at least the path reward function, the dynamic collision penalty function, and the step penalty function.

[0023] The above method, through the construction of a 3D grid map and obstacle detection, can construct real-time identification and labeling of tunnel interior walls and moving and static obstacles, and then construct an environmental state vector representing the real-time environment inside the tunnel. By combining the real-time state vector of the robotic arm, a joint state vector can be obtained, realizing a unified representation of the robotic arm state and the tunnel environment state. Based on the pre-trained robotic arm motion path model inside the tunnel, and based on path reward, dynamic collision penalty and step reward, it adapts to the motion characteristics of dynamic obstacles, thereby enabling rapid response to changes in the position of dynamic obstacles, effectively avoiding the risk of collision with dynamic obstacles, and better fitting the dynamic construction scenario of tunnels.

[0024] In step S101, the tunnel environment data includes point cloud data and image information. An anti-dust lidar (1550nm wavelength for stronger dust penetration) is used, and a pulse-time-of-flight algorithm reduces dust scattering interference. An infrared camera equipped with an active illumination module penetrates dust to obtain clear outlines. Point cloud data and image information within the tunnel are collected in real time. Based on this data, the tunnel wall outline, obstacle locations, and obstacle types are identified. Obstacle types include static or dynamic obstacles such as construction equipment and collapsed material piles. Furthermore, dust concentration is inferred from image grayscale values.

[0025] In step S102, the point cloud data and image information collected in step S101 are received. The point cloud data is filtered, denoised, and downsampled to remove redundant point clouds and abnormal noise points on the ground and tunnel walls, retaining target point clouds that may be obstacles. The image information is grayscaled and enhanced to improve the recognition of obstacle contours. At the same time, the point cloud data and image information are spatiotemporally registered to ensure that their spatial coordinates and timestamps are consistent. Based on the preprocessed point cloud data, a grid partitioning algorithm is used to divide the tunnel space into three-dimensional grid units according to a preset size. Each grid unit is marked with three states: idle, occupied, and unknown. The occupied grid is the area where obstacles may exist. Combined with the contour features of the tunnel wall, the occupied grid corresponding to the tunnel wall itself is removed, completing the basic construction of the real-time three-dimensional grid map of the tunnel wall.

[0026] Real-time detection of obstacles within the tunnel includes: (1) Perform cluster analysis on the grid areas marked as “occupied” in the map, and classify adjacent “occupied” grids into a target cluster. Each target cluster is an obstacle to be identified. At the same time, combine the image information, identify the outline of the obstacle in the image through the target detection algorithm, and match it with the target cluster obtained by point cloud clustering to confirm whether the obstacle actually exists. If so, preliminarily determine the location range of the obstacle.

[0027] (2) For obstacles initially detected, continuously track them: If the change in the location range of an obstacle is less than the preset static threshold, and the obstacle outline remains unchanged in conjunction with the image recognition results, it is determined to be a static obstacle. Common static obstacles in tunnel scenarios include temporary material stockpiles, fixed support structures, and idle construction equipment. After identification, their type identifiers are marked, and their coordinate range is fixed in the grid map.

[0028] If the change in the position range of an obstacle exceeds a preset static threshold and the movement speed remains stable, and the obstacle outline has motion blur and has a continuous displacement trajectory, based on the image recognition results, it is determined to be a dynamic obstacle. Common dynamic obstacles in tunnel scenarios include grouting machines, transport vehicles, construction workers, etc. After recognition, their type is marked, their position coordinates and movement speed are recorded in real time, and updated synchronously to the grid map.

[0029] The identified static and dynamic obstacles are marked on a 3D grid map, with the marking information including obstacle type identifier, real-time location coordinates, and movement speed; the marking information for all obstacles is also output synchronously. As an example, the type identifiers for grouting machines, transport vehicles, and temporary material stockpiles are marked as 1, 2, and 3, respectively.

[0030] Simultaneously, safe operating areas and high-risk restricted areas are marked on the map. High-risk restricted areas refer to areas where entering them could immediately cause a safety accident, such as the impact zone of temporary material collapse, the dynamic range of large construction equipment, and danger zones designated on tunnel walls due to structural instability. In path planning, these areas are set as absolutely prohibited, and any path intersecting with them will be directly rejected. The specific calculation method is as follows: (1) Collapse impact zone of temporary stockpiles: Based on the actual location range of the detected temporary stockpiles (static obstacles, IDtype=3), combined with the stockpile height preset collapse diffusion coefficient (e.g., when the stockpile height is ≤2m, the diffusion coefficient is 1.5), take the temporary stockpile outline boundary as the benchmark, and uniformly expand the corresponding distance in all directions. The expanded grid area is the collapse impact zone and is designated as a high-risk restricted area; (2) Dynamic activity range of large construction equipment: For the detected large dynamic obstacles (e.g., grouting machine IDtype=1, transport vehicle IDtype=2), combined with the equipment's own size (preset equipment parameter library) and dynamic motion characteristics, the dynamic restricted area is defined: In the static state, the restricted area is defined by extending the equipment outline boundary outward by a preset safety distance; in the dynamic movement, based on the real-time movement speed and movement trajectory prediction of the equipment, an additional buffer area in front of the trajectory is added, and the restricted area grid is updated in real time to ensure coverage of all possible areas reached by the equipment. (3) Tunnel wall structure instability danger zone: The tunnel wall cracks, spalling, bulging and other structural instability features are detected by image recognition algorithm; for the detected structural instability area, the pre-set safety margin is extended into the tunnel based on the outline of the unstable area, and the extended area is designated as a high-risk restricted area.

[0031] In step S103, the joint state vector includes the real-time state vector of the robotic arm and the state vector of the tunnel environment. The dimensions of the real-time state vector of the robotic arm include: the real-time pose of the robotic arm, the distance between the end effector and the fixed obstacle, and the distance between the end effector and the target point; the dimensions of the tunnel environment state vector include at least: obstacle position, obstacle velocity, and obstacle type identifier.

[0032] The real-time pose of the robotic arm includes real-time joint angles, joint angular velocities, and end effector posture, calculated using DH parameters. Specifically, rotational angle data of each joint of the robotic arm is collected in real time using joint sensors, serving as the real-time angles of each joint. For example, for a 6-DOF robotic arm, the real-time rotational angles of the six joints from θ1 to θ6 are obtained. Secondly, the real-time angular velocity of each joint is calculated by dividing the difference between two adjacent frames of joint angle data by the time interval between the two frames, ensuring that the angular velocity data is updated synchronously with the joint angle data. Finally, based on the preset DH parameters of the robotic arm and the calculated real-time angles of each joint, the posture angles (roll, pitch, yaw) and real-time 3D coordinates of the end effector are derived step by step using the DH forward kinematics formula. These, along with the real-time joint angles and joint angular velocities, constitute the current pose of the robotic arm. Mapping the coordinate system of the robotic arm base to the tunnel coordinate system transforms the real-time 3D coordinates (x, y, z) of the end effector from the robotic arm coordinate system to the tunnel coordinate system.

[0033] After obtaining the real-time 3D coordinates (x, y, z) of the end effector, based on the obstacle positions and the target point positions in the 3D grid map, the spatial Euclidean distance calculation method is used to calculate the straight-line distance between the real-time 3D coordinates of the end effector and the nearest point on the obstacle boundary contour. This distance is the distance between the end effector and the fixed obstacle. Similarly, the straight-line distance between the real-time 3D coordinates of the end effector and the target point position is calculated and denoted as the distance between the end effector and the target point.

[0034] In some embodiments, the dimensions of the tunnel environment state vector also include: dust concentration and a set of key geometric feature points on the tunnel inner wall.

[0035] Dust concentration is detected based on the image in front. Specifically, an image-based dust concentration detection model is established in advance to obtain real-time images in front. The current dust concentration is estimated based on the dust concentration detection model. The training method of the dust concentration detection model is as follows: (1) Collect sample images of different dust concentration scenarios in the tunnel. Simultaneously, collect the standard dust concentration values ​​(quantized to the 0-1 range) of the corresponding scenarios through a dedicated dust sensor to form a training dataset of image-concentration pairs. The samples need to include tunnel images with different lighting and different working positions. (2) Preprocess the sample images in the training dataset to complete grayscale conversion, filtering, noise reduction and enhancement. Extract the visual features of the images, including image clarity, contrast, grayscale mean and edge blur. Construct feature vectors based on the visual features and associate the feature vectors with the corresponding dust concentration values. (3) Train the dust concentration detection model based on a deep learning model, such as a CNN convolutional neural network, with the feature vector as input and the dust concentration value as output. The model is iteratively trained by setting a loss function based on the deviation between the model's predicted concentration and the standard concentration. In each training round, the model predicts the corresponding dust concentration value based on the visual feature vector of the input image, calculates the deviation between the predicted value and the standard value through the loss function, and updates the model parameters by backpropagation using the optimizer to gradually reduce the prediction deviation.

[0036] The obstacle location, obstacle velocity, obstacle type identifier, and key geometric feature point set of the tunnel inner wall in the tunnel environment state vector are obtained based on a 3D raster map. Specifically, if the obstacle is static, its velocity is recorded as 0. The key geometric feature point set of the tunnel inner wall prioritizes the contour points of key areas such as narrow passages, passing tunnels, and tunnels with variable cross-sections. Straight sections and flat inner walls are not extracted. All key feature points are integrated in the format of "coordinate vector + area identifier" to form a feature point set: S_contour = {[x1,y1,z1,id1], [x2,y2,z2,id2], ...}. This set is updated synchronously with the raster map. If there are slight changes in the contour of a key area, such as temporary support offset, the coordinates of the corresponding feature points are immediately updated to ensure consistency with the actual inner wall state of the tunnel.

[0037] In step S104, the motion path model of the robotic arm in the tunnel is trained using a two-stage training method: pre-training of a general robotic arm path planning model and fine-tuning of a robotic arm path planning model specifically for tunnel operations. Specifically, step S104 includes: S1041. Using a basic state vector and combining it with the robotic arm's motion space, a reinforcement learning model is used to pre-train the robotic arm's path planning model. During the training process, the reward function used includes at least a path reward function, a collision penalty function, and a step penalty function.

[0038] Specifically, a general industrial simulation environment free from dust interference and dynamic sudden obstacles is constructed to simulate the basic operation scenario of the robotic arm; at the same time, basic state vector data is acquired and spatiotemporally registered and normalized. The basic state vector contains only the real-time state vector of the robotic arm and basic environmental features. The real-time state vector of the robotic arm includes: the current pose of the robotic arm, the distance between the end effector and a fixed obstacle, the distance between the end effector and the target point, and the basic environmental features including the position of static obstacles.

[0039] The robotic arm motion space refers to the set of all quantized and discretized motion commands that the robotic arm can actively execute. It is the object of the DQN reinforcement learning model's output decision. Based on the robotic arm's motion capabilities, the motion space includes two types of discretized motion commands: one type is joint motion commands, designed for each joint, including independent discrete actions such as fixed-angle clockwise / counterclockwise rotation (e.g., 1°, 2°) and joint stillness. Each joint corresponds to a dedicated action option, allowing for flexible adjustment of the overall robotic arm posture. The other type is end-effector displacement adjustment commands, including fixed-distance movement of the end-effector along the x, y, and z axes of the tunnel coordinate system (e.g., 5mm, 10mm), as well as small pitch and rotational attitude fine-tuning commands. All discrete actions correspond to a unique action identifier, one-to-one with the Q-value output by the DQN algorithm, ensuring that the model can quickly match executable actions by filtering for the optimal Q-value and convert them into robotic arm pose adjustment parameters.

[0040] In response to the operational requirements of dusty tunnel environments, some embodiments also use dust concentration as a constraint on the robotic arm's motion space, dynamically limiting the executable range of the motion space: the higher the dust concentration, the stricter the constraint on the robotic arm's motion space, and the lower the corresponding upper limit of its movement speed. Specifically, by setting a mapping relationship between dust concentration and the upper limit of the robotic arm's movement speed, for example, in high-dust scenarios (such as C≥200mg / m³), the angular velocity of the robotic arm's joints is directly limited to 70%, thereby reducing dust emissions and ensuring the stability of the robotic arm's operation within the tunnel.

[0041] A DQN reinforcement learning model is initialized, taking the basic state vector as input and the Q-values ​​corresponding to discrete actions in the robotic arm's motion space as output. A pre-training dataset is constructed, where the current pose of the robotic arm in the input data is used by the model to perceive its own state, the distance between the end effector and the obstacle / target point is used by the model to determine the direction of motion, and the basic obstacle positions are used by the model to learn basic obstacle avoidance logic. Using the above training dataset, the basic path planning model is iteratively trained based on the DQN reinforcement learning model. During training, a basic reward function is used to evaluate the quality of actions. Specifically, in each training step, the model receives the basic joint state vector at the current time, approximates the Q-value function through the evaluation network, and outputs the Q-values ​​corresponding to each discrete action. An ε-greedy strategy is used to select the action corresponding to the optimal Q-value, and this action is applied to the pre-training simulation environment. The environment calculates the reward value based on the action execution result, combined with the basic reward function, using the distance parameter in the input data. If the action shortens the distance between the end effector and the target point, a positive reward is given; if an obstacle is touched, a negative penalty is given. The interaction data of the state vector, action, reward, and next state vector corresponding to each step is stored in the experience replay buffer for model parameter optimization.

[0042] S1042. Using the tunnel joint state vector and combining it with the robotic arm motion space, the pre-trained robotic arm path planning model is iteratively fine-tuned. During the fine-tuning process, the penalty function used includes at least the path reward function, the dynamic collision penalty function, and the step penalty function.

[0043] The method for constructing the joint state vector is the same as in step S103, and will not be repeated here. The joint state vector is based on the basic joint state vector with the addition of a tunnel-specific environment dimension, including the real-time state vector of the robotic arm and tunnel environment data. The tunnel environment data includes: obstacle position, obstacle speed, and obstacle type identifier.

[0044] In some embodiments, the reward function also includes a dust concentration calculation stability detection reward and a narrow passage reward. In this case, the tunnel environmental data also includes: dust concentration and a set of key geometric feature points on the tunnel inner wall.

[0045] The simulation environment incorporates unique tunnel-specific elements, including complex dynamic disturbances such as dust fluctuations, sudden obstacles, and narrow passages. Key geometric feature points on the tunnel wall (narrow passages, passing bays, etc.) are spatially correlated with the real-time state vector of the robotic arm. Obstacle speed and type identifiers are bound to obstacle positions, and dust concentration is correlated with the robotic arm's pose adjustment threshold, ensuring that the input data reflects the dynamic changes of the tunnel scene. The weights of the pre-trained model converged in the first stage are transferred to this stage, adapted to a tunnel-specific reward function. Dynamic collision penalties are calculated using obstacle speed and type identifiers from the input data (penalty doubles when obstacles are close, and penalty weights are increased for high-priority obstacles). Stability detection rewards are calculated using dust concentration (high dust levels suppress large robotic arm movements, providing positive rewards for smooth movements). Narrow passage rewards are calculated using key geometric feature points on the tunnel wall and end-effector distance parameters (positive rewards are given when the trajectory is close to the centerline). Posture rationality rewards are calculated using the robotic arm's current pose, allowing the reward function to accurately evaluate the quality of actions in the tunnel scene based on feedback from the input data. The core principle of each training step is consistent with the pre-training phase. By evaluating the Q-values ​​of each discrete action output by the network, the ε-greedy strategy is optimized in combination with the requirements of the tunnel scenario. After the action is executed, the environment calculates the reward value using the real-time update of the complete input data and feeds it back to the model. The model samples interaction data from the experience replay buffer and optimizes the Q-value function to approximate the accuracy through parameter updates of the target network and the evaluation network, allowing the model to gradually learn the tunnel-specific decision rules.

[0046] In step S1042, the tunnel-specific reward function is used in the fine-tuning phase. Path rewards Dynamic collision penalty Stability testing Step count penalty Narrow passage reward It consists of five parts, each with an initial weight, which can be dynamically adjusted based on obstacle priority. The specific expression is as follows:

[0047] The specific definitions and calculation methods of each sub-item reward / penalty function are as follows: a. Path reward function: The purpose is to guide the algorithm toward the goal. In the formula... This is the path ratio coefficient, and also serves as the initial weight for path rewards. and These represent the distances between the end effector of the robotic arm and the target point before and after the action is performed. If the distance between the end effector and the target point is shortened after the action is performed, a positive reward is generated. The more the distance is shortened, the larger the positive reward value.

[0048] b. Dynamic collision penalty function: This function is a piecewise function, specifically:

[0049] If the robotic arm collides, R_c ollision =C (where C is a large negative number), inflict a severe penalty; if no collision occurs, R_c ollision =0, no reward or penalty is given. Collisions between the robotic arm and the robot in underground tunnels can easily cause safety accidents, therefore strict collision penalties are necessary; additionally, a penalty based on obstacle movement trend prediction is added. If an obstacle is detected approaching the robotic arm, the collision penalty is doubled (i.e., R_c). ollision =2C), further enhancing the obstacle avoidance safety of the robotic arm. In addition, a collision detection and path rejection mechanism is set up in conjunction with high-risk restricted areas: the model will perform preliminary path collision detection, and any candidate path that crosses the high-risk restricted area (red grid) will be directly assigned an extremely low reward value (or an extremely high penalty), thus being effectively eliminated in the decision-making stage to prevent the robotic arm from entering the danger zone.

[0050] c. Stability detection function: The aim is to ensure the robotic arm operates stably and continuously in dusty environments, suppressing large movements to reduce dust generation. In the formula... is the stability detection coefficient, and also serves as the initial weight for the stability reward. i is the acceleration of the robotic arm, and ∥i∥ is the magnitude of the acceleration. The smoother the robotic arm's movement (the smaller the acceleration), the closer ∥i∥ is to 0, and the larger the stability detection reward value. If the movement is violent (the greater the acceleration), the closer ∥i∥ is to 1, the smaller the reward value, or even close to 0, thus guiding the model to output smooth movements.

[0051] d. Step penalty function: The aim is to encourage the robotic arm to complete its task as quickly as possible, avoiding ineffective movements and redundant steps. In the formula, n represents the current number of steps. The preset maximum number of steps, This is the step penalty coefficient (negative), which also serves as the initial weight for the step penalty. The closer the current step number n is to the maximum step number n... max The larger the absolute value of the step penalty, the more it encourages the model to converge to the optimal path quickly, thereby improving work efficiency.

[0052] e. Narrow channel reward function: The aim is to encourage the robotic arm to move close to the tunnel's centerline, maximizing operational safety margins. In the formula... The narrow passage reward coefficient also serves as the initial weight for the narrow passage reward. The path width is the minimum distance between the current movement trajectory of the robotic arm and the inner wall of the tunnel, and boundary constraints are added: the path width ≥ the preset safety margin (the preset safety margin is set according to the width of the robotic arm itself and the safety requirements of tunnel operation to avoid the path width approaching 0), to ensure that the adjustment effect of the weight is effective; the smaller the path width (i.e., the closer to the center line, the greater the safety margin), the greater the narrow passage reward value, guiding the robotic arm to operate safely in narrow passage scenarios.

[0053] K p K s K step K t All proportional coefficients are used as the initial weights of their respective sub-items, with the initial sum set to 1.

[0054] During the fine-tuning of tunnel scene migration, various types of obstacles exist in the tunnel environment (such as grouting machines, transport vehicles, temporary material stockpiles, etc.). Different obstacles have different degrees of impact on the safety of the robotic arm operation. If a reward function with fixed weights is used, it cannot reflect the differences in the safety priority of obstacles, which may lead to the robotic arm making non-optimal avoidance decisions under complex working conditions. Therefore, in some embodiments, the weights of each component of the reward function are dynamically adjusted by obstacle avoidance priority and dust concentration to ensure that the robotic arm prioritizes avoiding high-risk obstacles while taking into account work efficiency. Based on this, a dynamic adjustment module for the reward function is set up, and the weight adjustment of each component is carried out based on the above-mentioned initial weights. Specifically, an obstacle priority mapping table is set up, which includes the mapping relationship between obstacle type and priority coefficient. When one or more obstacles are detected, the weights of each component in the reward function are adjusted according to the coefficient of the obstacle with the highest priority. The adjustment rule is: when the obstacle avoidance weight increases, the other weights decrease simultaneously. Specifically, the weights corresponding to the narrow passage reward item and the dust environment reward item are compressed first, while the weights corresponding to the path reward item and the step penalty item are retained as much as possible. The sum of the weights should be approximately equal to 1, for example, [0.8, 1.2]. Set an adjustment threshold for the dust environment stability detection function. When the dust concentration is lower than the set low dust threshold, it can be considered that the dust has no effect on the control of the robotic arm. Reduce the weight of the stability detection function and increase the weight of other functions so that the sum of the weights of each function meets the requirements.

[0055] Obstacle priorities are set based on the tunnel construction task (e.g., priority avoidance of grouting machines, followed by transport vehicles). By dynamically adjusting the weights of each reward function, the path planning target is updated in real time to ensure that the robotic arm prioritizes avoiding high-risk obstacles. For example, the mapping table is: IDtype=1 (grouting machine), P=1.0 (highest priority); IDtype=2 (transport vehicle), P=0.8; IDtype=3 (temporary material stockpile), P=0.5. In the event of a sudden material collapse (IDtype=3, priority coefficient P=0.5), the initial weight (corresponding to the collision penalty weight) is temporarily increased to 1.5 times using a weight scaling factor λ, while simultaneously prioritizing the reduction of the weights for narrow passage rewards and dust environment rewards, ensuring a weight sum of 1.0. In low-dust scenarios (C < 100 mg / m³), the weight ratio of the stability detection function is reduced, while the path reward weight is appropriately increased, maintaining a weight sum of around 1.0 to improve the robotic arm's movement efficiency and achieve a dynamic balance between safety and efficiency.

[0056] The aforementioned reward functions are specifically designed to address the unique needs of tunnel scenarios, adapting to dusty environments, narrow passages, and dynamic obstacles. The path reward function guides the robotic arm toward the work target, reducing invalid paths and improving work targeting. The dynamic collision penalty function avoids collisions between the robotic arm and obstacles or high-risk areas, aligning with the safety-first requirements of tunnel construction. It also adjusts the penalty weight based on obstacle priority, further strengthening the avoidance guidance for high-risk obstacles. The stability detection function guides the robotic arm to output smooth movements, effectively suppressing dust generated by large movements, adapting to the operational requirements of dusty tunnel scenarios. Furthermore, since dust concentration is a constraint on the motion space, it ensures operational stability in dusty environments. The step penalty function reduces redundant movements and invalid steps, improving the robotic arm's operational efficiency. The narrow passage reward function guides the robotic arm to maximize operational safety margins in narrow passage scenarios, adapting to the characteristics of confined tunnel spaces. It can quickly respond to complex scene changes such as dynamic obstacles, dust fluctuations, and narrow passages within the tunnel, outputting efficient, safe, stable, and scenario-appropriate path decisions.

[0057] After obtaining the pose adjustment parameters of the robotic arm, the control signal is transmitted to the actual tunnel construction robot. The state space and path planning model are continuously adjusted based on real-time environmental changes, thereby optimizing subsequent path planning decisions and achieving path feedback adjustment. Online model updates are implemented; when the LiDAR detects a sudden obstacle, local path replanning is triggered, with a control cycle ≤50ms, ensuring the robotic arm completes the avoidance maneuver within 0.5s.

[0058] Figure 2A schematic diagram of an apparatus provided by one or more embodiments of the present invention is shown. The apparatus 200 includes: a real-time data acquisition module 201, configured to acquire real-time environmental data within a tunnel and real-time posture parameters of a robotic arm; a grid map construction module 202, configured to construct a real-time three-dimensional grid map of the tunnel wall based on the tunnel environmental data, simultaneously detect obstacles within the tunnel in real-time, and mark the detected obstacles on the three-dimensional grid map; a state vector construction module 203, configured to construct a real-time state vector of the robotic arm based on its real-time posture parameters, construct a tunnel environment state vector based on the tunnel wall and obstacles, and construct a real-time joint state vector based on the real-time state vector of the robotic arm and the tunnel environment state vector; and a dynamic path planning module 204, configured to obtain the pose adjustment parameters of the robotic arm based on the real-time joint state vector and a pre-trained motion path model of the robotic arm within the tunnel; wherein the motion path model of the robotic arm within the tunnel is trained based on a reinforcement learning model, and the penalty function includes at least a path reward function, a dynamic collision penalty function, and a step penalty function.

[0059] Furthermore, one or more embodiments of the present invention also provide an electronic device that can be used to implement the dynamic path planning method for a tunnel construction robotic arm in the above embodiments. The electronic device includes one or more processors, one or more memories coupled to the processors, and a communication module coupled to the processors.

[0060] The memory may include one or more non-volatile memories and one or more volatile memories. Examples of non-volatile memories include, but are not limited to, at least one of the following: read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, hard disk, compact disc (CD), digital video disc (DVD), or other magnetic and / or optical storage. Examples of volatile memories include, but are not limited to, at least one of the following: random access memory (RAM) or other volatile memories that do not persist during the duration of a power outage. The computer program may be stored in the ROM. When the processor executes the computer program, it implements the above-described dynamic path planning method for the tunnel construction robot arm.

[0061] In some embodiments, the program may be tangibly contained in a computer-readable medium, which may include a device (such as a memory) or other storage device accessible by the device. The program may be loaded from the computer-readable medium into RAM for execution. The computer-readable medium may include any type of tangible non-volatile memory, such as ROM, EPROM, flash memory, hard disk, where a computer program is stored that, when executed by a processor, implements the aforementioned dynamic path planning method for a tunnel construction robotic arm.

[0062] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a server or terminal, they generate, in whole or in part, the processes or functions described in the embodiments of this application. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic cable, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to the server or terminal, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, and magnetic tape), an optical medium (e.g., digital video disk (DVD), etc.), or a semiconductor medium (e.g., solid-state drive).

[0063] Furthermore, although the operations are described in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this application. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0064] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A dynamic path planning method for a tunnel construction robotic arm, characterized in that, Includes the following steps: Real-time acquisition of environmental data inside the tunnel and real-time attitude parameters of the robotic arm; Based on the environmental data inside the tunnel, a real-time three-dimensional grid map is constructed on the inner wall of the tunnel. At the same time, obstacles inside the tunnel are detected in real time, and the detected obstacles are marked on the three-dimensional grid map. A real-time state vector of the robotic arm is constructed based on its real-time posture parameters, a tunnel environment state vector is constructed based on the tunnel wall and obstacles, and a real-time joint state vector is constructed based on the real-time state vector of the robotic arm and the tunnel environment state vector. Based on the real-time joint state vector, the pose adjustment parameters of the robotic arm are obtained based on the pre-trained motion path model of the robotic arm in the tunnel. The motion path model of the robotic arm in the tunnel is trained based on a reinforcement learning model, and the penalty function includes at least the path reward function, the dynamic collision penalty function, and the step penalty function.

2. The dynamic path planning method for a tunnel construction robotic arm as described in claim 1, characterized in that, Real-time detection of obstacles within the tunnel includes: Real-time acquisition of point cloud data and image information within the tunnel; identification of target point clouds that may be obstacles from the point cloud data; Based on point cloud data, the tunnel space is divided into three-dimensional grid units according to a preset size. Each grid unit is marked with three states: free, occupied, and unknown. The occupied grid is the area where obstacles may exist. Cluster analysis is performed on the grid areas marked as occupied in the map, and each target cluster is a barrier to be identified; the outline of the barrier is identified based on image information, and the authenticity of the barrier is confirmed by position matching with the barrier to be identified; The system continuously tracks initially detected obstacles and distinguishes between static and dynamic obstacles based on their location range and contour changes.

3. The dynamic path planning method for a tunnel construction robotic arm as described in claim 1, characterized in that, The dimensions of the robotic arm's real-time state vector include: the robotic arm's real-time pose, the distance between the end effector and the fixed obstacle, and the distance between the end effector and the target point; the dimensions of the tunnel environment state vector include at least: obstacle position, obstacle velocity, and obstacle type identifier.

4. The dynamic path planning method for a tunnel construction robotic arm as described in claim 3, characterized in that, The training method for the motion path model of the robotic arm inside the tunnel is as follows: Using a basic state vector and combining it with the robotic arm's motion space, a reinforcement learning model is used to pre-train the robotic arm's path planning model. During the training process, the reward function used includes at least a path reward function, a collision penalty function, and a step penalty function. The basic state vector includes the robotic arm's real-time state vector and the basic environment state vector, and the basic environment state vector includes the positions of static obstacles. By employing a tunnel joint state vector and combining it with the robotic arm's motion space, the pre-trained robotic arm path planning model is iteratively fine-tuned. During the fine-tuning process, the penalty functions used include at least the path reward function, the dynamic collision penalty function, and the step count penalty function.

5. The dynamic path planning method for a tunnel construction robotic arm as described in claim 4, characterized in that, The dimensions of the tunnel environment state vector also include: dust concentration and the set of key geometric feature points on the tunnel inner wall; during the iterative fine-tuning of the pre-trained robotic arm path planning model, the robotic arm action space is determined based on the dust concentration and the mapping relationship between the set dust concentration and the upper limit of the robotic arm's movement speed. The penalty function used also includes a stability detection function and a narrow passage reward function.

6. The dynamic path planning method for a tunnel construction robotic arm as described in claim 4 or 5, characterized in that, The total reward function used in the fine-tuning phase is the sum of all reward functions; The dynamic collision penalty function is a piecewise function. When the robotic arm collides with another object, the value is set; otherwise, the value is 0. The path reward function, step penalty function, stability detection function, and narrow channel reward function are all products of weights and reward terms; When one or more dynamic obstacles are detected, the weights of each item in the reward function are adjusted based on the obstacle priority mapping table and the coefficient of the highest priority obstacle. When the weight of the dynamic collision penalty function increases, the weights corresponding to the narrow passage reward function and the dust environment reward function are compressed first, while the weights corresponding to the path reward function and the step penalty function are retained, so that the sum of the weights meets the set value range. When the dust concentration is lower than the set low dust threshold, the weight of the stability detection function is reduced and the weights of other functions are increased, so that the sum of the weights meets the set value range.

7. A dynamic path planning device for a tunnel construction robotic arm, characterized in that, include: The real-time data acquisition module is configured to collect real-time environmental data inside the tunnel and the real-time attitude parameters of the robotic arm. The raster map building module is configured to build a real-time three-dimensional raster map of the tunnel wall based on the tunnel environment data, and at the same time detect obstacles in the tunnel in real time and mark the detected obstacles in the three-dimensional raster map. The state vector construction module is configured to construct a real-time state vector of the robotic arm based on the real-time attitude parameters of the robotic arm, construct a tunnel environment state vector based on the tunnel wall and obstacles, and construct a real-time joint state vector based on the real-time state vector of the robotic arm and the tunnel environment state vector. The dynamic path planning module is configured to obtain the pose adjustment parameters of the robotic arm based on the real-time joint state vector and the pre-trained motion path model of the robotic arm in the tunnel. The motion path model of the robotic arm in the tunnel is trained based on a reinforcement learning model, and the penalty function includes at least a path reward function, a dynamic collision penalty function, and a step penalty function.

8. An electronic device, characterized in that, It includes a processor and a memory, the memory storing computer instructions that, when executed by the processor, cause the electronic device to perform the method of any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method of any one of claims 1 to 6.

10. A computer program product, the computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images