Obstacle avoidance path planning method of building robot and related equipment

Abstract

The invention provides a building robot obstacle avoidance path planning method and related equipment, and the method comprises the steps: fusing the state information of a target building robot with environment data through Bayesian filtering, and obtaining an occupied grid map; inputting the occupied grid map into the trained lightweight convolutional neural network for feature extraction, and generating environment feature data; the environment feature data and the current state of the target building robot are input into a trained decision model for decision making, an action instruction is obtained, the target building robot reaches the target position within the shortest time based on the action instruction, and the path from the target building robot to the target position is the obstacle avoidance path of the target building robot; by improving the accuracy and robustness of environment perception, efficient obstacle avoidance and path planning in a complex environment are realized.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an obstacle avoidance path planning method for a construction robot and related equipment. Background Art

[0002] With the rapid development of the construction industry, the demand for automation and intelligence at construction sites is increasing. Construction robots are increasingly used in material distribution, inspection, cleaning, etc. at construction sites. However, the construction site environment is complex and changeable, with a large number of dynamic and static obstacles, such as construction workers, machinery and equipment, and material stacking. At the same time, the construction site environment usually has problems such as insufficient light, high dust, and many obstructions, which makes it difficult for construction robots based on visual sensors (such as cameras) to reliably perceive the environment.

[0003] Traditional path planning algorithms, such as The algorithm and Dijkstra algorithm are mainly based on known static maps and are difficult to adapt to changes in dynamic environments. In addition, these algorithms mainly target the shortest path length and do not fully consider the total time required to complete the task. Traditional obstacle avoidance algorithms, such as the artificial potential field method, may produce "local minimum" problems, causing the robot to get into trouble, and are prone to oscillation in complex environments, affecting task efficiency.

[0004] Although the path planning method based on deep reinforcement learning has certain environmental adaptability, it usually does not specifically optimize the task completion time, which may result in a lengthy path or a large time consumption. Existing methods still have shortcomings in real-time performance, obstacle avoidance performance and task efficiency, and cannot meet the needs of construction robots to complete tasks in the shortest time in complex dynamic environments. Summary of the invention

[0005] The present invention provides an obstacle avoidance path planning method and related equipment for a construction robot, the purpose of which is to achieve efficient obstacle avoidance and path planning in a complex environment.

[0006] In order to achieve the above object, the present invention provides an obstacle avoidance path planning method for a construction robot, comprising: Step 1, obtaining the environmental data of the target space; Step 2: The state information of the target construction robot is fused with the environmental data through Bayesian filtering to obtain an occupancy grid map; Step 3, input the occupancy grid map into the trained lightweight convolutional neural network for feature extraction to generate environmental feature data; Step 4: Input the environmental feature data and the current state of the target construction robot into the trained decision model to make a decision and obtain an action instruction. The target construction robot reaches the target position in the shortest time based on the action instruction. The path taken by the target construction robot to the target position is the obstacle avoidance path of the target construction robot.

[0007] More specifically, step 1 includes: Setting up a laser radar and multiple ultrasonic sensors inside the target construction robot; Acquire three-dimensional point cloud data of the target space through laser radar; Acquire the distance data between the target construction robot and obstacles through multiple ultrasonic sensors; Integrate 3D point cloud data and distance data into environment data of target space.

[0008] Furthermore, before fusing the state information of the target construction robot with the environmental data through Bayesian filtering, it also includes: Preprocessing the environmental data to obtain preprocessed environmental data; The state information of the target construction robot is fused with the preprocessed environmental data through Bayesian filtering data.

[0009] More specifically, step 2 includes: Obtain the position, movement speed and action instructions of the target construction robot at the previous moment; Based on the position, motion speed and action command of the target construction robot at the previous moment, a kinematic model is used to generate a priori state estimation, which includes the current position, speed and steering information of the target construction robot; The preprocessed environmental data is fused with the prior state estimate to obtain the posterior state estimate; The posterior state estimate is mapped onto a two-dimensional grid map to obtain an occupancy grid map.

[0010] Furthermore, the computational expression for the posterior state estimate is: in, represents the posterior state estimate, represents the prior state estimate, represents the Kalman gain, Indicates time The observed value of represents the observation matrix, express The transposed matrix of represents the observation noise covariance matrix, represents the identity matrix, represents the posterior state estimate covariance matrix, represents the prior state covariance matrix.

[0011] Specifically, the trained lightweight convolutional neural network includes: The input layer, the first convolutional layer, the first pooling layer, the second convolutional layer, the second pooling layer, the fully connected layer and the output layer are connected in sequence; The input layer is used to receive the occupancy grid map; The first convolutional layer and the second convolutional layer are both used to extract features from the occupancy grid map; The first pooling layer and the second pooling layer are both used to downsample the extracted features; The fully connected layer is used to flatten the features output by the second pooling layer to obtain the environment feature vector; The output layer generates environmental feature data based on the environmental feature vector.

[0012] Further, the trained decision model includes a policy network and a value network; The policy network generates action probability distribution based on the environmental feature data and the current state of the target construction robot;

[0013] The value network generates state value estimates based on the environmental feature data and the current state of the target construction robot.

[0014] The present invention also provides an obstacle avoidance path planning device for a construction robot, comprising: An acquisition module is used to obtain the environmental data of the target space; A fusion module is used to fuse the state information of the target construction robot with the environmental data through Bayesian filtering to obtain an occupancy grid map; An extraction module is used to input the occupancy grid map into the trained lightweight convolutional neural network for feature extraction to generate environmental feature data;

[0015] The decision-making module is used to input the environmental feature data and the current state of the target construction robot into the trained decision-making model to make a decision and obtain an action instruction. The target construction robot reaches the target position in the shortest time based on the action instruction. The path taken by the target construction robot to the target position is the obstacle avoidance path of the target construction robot.

[0016] The present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, an obstacle avoidance path planning method for a construction robot is implemented.

[0017] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the obstacle avoidance path planning method of the construction robot is implemented.

[0018] The above scheme of the present invention has the following beneficial effects: The present invention obtains environmental data of the target space; fuses the state information of the target construction robot with the environmental data through Bayesian filtering to obtain an occupied grid map; inputs the occupied grid map into a trained lightweight convolutional neural network for feature extraction to generate environmental feature data; inputs the environmental feature data and the current state of the target construction robot into a trained decision model for decision-making to obtain action instructions, and the target construction robot reaches the target position in the shortest time based on the action instructions, and the path taken by the target construction robot to the target position is the obstacle avoidance path of the target construction robot; compared with the prior art, the present invention first uses Bayesian filtering to fuse the state information of the target construction robot with the environmental data, thereby improving the accuracy and robustness of environmental perception, uses a lightweight convolutional neural network and an optimized reinforcement learning network to extract data features and generate action instructions, thereby reducing computational complexity, improving decision-making speed, and meeting real-time requirements, and finally realizes efficient obstacle avoidance and path planning in complex environments based on action instructions.

[0019] Other beneficial effects of the present invention will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of a flow chart of an embodiment of the present invention; Figure 2 Schematic diagram of the structure of the obstacle avoidance path planning device in an embodiment of the present invention; Figure 3 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] In the description of the present invention, it should be noted that the terms “first”, “second” and “third” are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0023] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a locking connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0024] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0025] The target space mentioned in the embodiment of the present invention is an actual construction site environment. The target position mentioned in the embodiment of the present invention is the construction point where the materials are transported. The construction site environment is complex and there are dynamic obstacles (such as construction workers and vehicles with uncertain trajectories) and static obstacles (such as material piles, equipment, waste piles, etc.). The purpose of the obstacle avoidance path planning method for a construction robot provided in the embodiment of the present invention is to plan a path that can safely avoid obstacles in the shortest time to complete the material delivery task.

[0026] In view of the above-mentioned tasks, the present invention provides an obstacle avoidance path planning method and related equipment for a construction robot.

[0027] like Figure 1 As shown, an embodiment of the present invention provides an obstacle avoidance path planning method for a construction robot, comprising: Step 1, obtaining the environmental data of the target space; Step 2: The state information of the target construction robot is fused with the environmental data through Bayesian filtering to obtain an occupancy grid map; Step 3, input the occupancy grid map into the trained lightweight convolutional neural network for feature extraction to generate environmental feature data; Step 4: Input the environmental feature data and the current state of the target construction robot into the trained decision model to make a decision and obtain an action instruction. The target construction robot reaches the target position in the shortest time based on the action instruction. The path taken by the target construction robot to the target position is the obstacle avoidance path of the target construction robot.

[0028] Most preferably, step 1 comprises: Setting up a laser radar and multiple ultrasonic sensors inside the target construction robot; Acquire three-dimensional point cloud data of the target space through laser radar; Acquire the distance data between the target construction robot and obstacles through multiple ultrasonic sensors; Integrate 3D point cloud data and distance data into environment data of target space.

[0029] In an embodiment of the present invention, the laser radar (LiDAR) equipped with the target construction robot is a 16-line laser radar, model Velodyne VLP-16, with a horizontal scanning range of 360°, a vertical scanning range of 30°, a ranging range of 0.2 meters to 100 meters, a horizontal angular resolution of 0.2°, and a vertical angular resolution of 1°. It is used to obtain three-dimensional point cloud data of the surrounding environment of the target construction robot in the target space, and the surrounding environment refers to the space with a distance greater than 0.2 meters and less than 100 meters from the target construction robot; since there are still some dynamic obstacles in the target space, relying solely on laser radar to obtain environmental data is likely to lead to low accuracy in close-range obstacle detection, resulting in a "darkness under the lamp" situation.

[0030] Based on the above situation, the embodiment of the present invention installs multiple ultrasonic sensors in front and on the side of the target construction robot to obtain the distance data between the target construction robot and the obstacle, so as to cover the short-range range and make up for the shortcomings of the lidar in the detection of close-range obstacles. The close-range range refers to the space where the distance from the target construction robot is greater than 0.02 meters and less than 5 meters. The model of the ultrasonic sensor is HC-SR04. The embodiment of the present invention requires at least three ultrasonic sensors.

[0031] It should be noted that the target construction robot adopts differential drive and has precise motion control capabilities.

[0032] Most preferably, before fusing the state information of the target construction robot with the environmental data through Bayesian filtering, the method further includes: Preprocessing the environmental data to obtain preprocessed environmental data; The state information of the target construction robot is fused with the preprocessed environmental data through Bayesian filtering data. The state information of the target construction robot includes position, movement speed and action instructions. The action instructions include forward speed and steering angle.

[0033] In an embodiment of the present invention, data preprocessing includes: 1. Preprocess the 3D point cloud data and distance data separately, as follows: For 3D point cloud data: Firstly, median filtering is used for denoising to obtain denoised point cloud data; Then, the point cloud data after denoising is downsampled using the voxel grid filter stack to obtain downsampled point cloud data; Then the downsampled point cloud data is converted from the sensor coordinate system to the robot coordinate system; Finally, the point cloud data in the robot coordinate system is converted to the global coordinate system to obtain the point cloud data in the global coordinate system; For distance data: First, the distance data is checked for validity, and valid distance data is retained; Then, a sliding average filter is applied to filter the effective distance data to obtain filtered distance data; Then the filtered distance data is converted from the sensor coordinate system to the robot coordinate system; Finally, the distance data in the robot coordinate system is converted to the global coordinate system to obtain the distance data in the global coordinate system; Second, interpolate the 3D point cloud data and distance data in the global coordinate system to ensure that the data from the lidar and ultrasonic sensors are synchronously aligned on the same time basis and match the sampling rates of different sensors, thereby providing high-quality input for subsequent multi-sensor data fusion and environmental modeling.

[0034] Specifically, step 2 includes: Obtain the position, movement speed and action instructions of the target construction robot at the previous moment; Based on the position, motion speed and action command of the target construction robot at the previous moment, a kinematic model is used to generate a priori state estimation, which includes the current position, speed and steering information of the target construction robot; The preprocessed environmental data is fused with the prior state estimate to obtain the posterior state estimate; The posterior state estimate is mapped onto a two-dimensional grid map to obtain an occupancy grid map.

[0035] In an embodiment of the present invention, the prediction and update steps of Bayesian filtering are used to estimate the environmental state. The prediction step and the update step are correlated in the Bayesian filtering process; the prediction step uses the known system model and control input to generate a priori estimate of the state at the current moment; the update step uses environmental data combined with the prior estimate to calculate a more accurate posterior state estimate.

[0036] Specifically, the prediction step provides the prior information required for the update step, and the update step corrects the prediction results to reflect the actual observed environmental changes. This iterative process enables the filter to continuously track and estimate the system state, improving the accuracy and reliability of environmental perception.

[0037] Specifically, the prediction steps: According to the position, speed and control input of the target construction robot at the previous moment, a kinematic model is constructed to predict the environmental state at the current moment. The expression of the kinematic model is: in, Represents the prior state estimate at the current moment, including information such as position, speed, and steering. Represents the state transfer function, which is used to describe the state of the target construction robot from time Evolution to the moment , based on the posterior state estimate and control input Prior state estimate at the prediction time , Represents the posterior state estimate at the previous moment, that is, moment, including the position, speed and orientation angle of the construction robot at the previous moment. Indicates the control input of the target construction robot at the previous moment, representing the control commands or actions imposed on the construction robot from the outside. These commands directly affect the state change of the construction robot. The control input includes the specific actions performed by the construction robot. , Indicates the current time The control input, Indicates that the construction robot is at time The acceleration command, Indicates that the construction robot is at time The steering angle change command, represents the prior estimated covariance matrix, represents the linear approximation of the state transfer matrix, express The transpose function of represents the process noise covariance matrix.

[0038] It is worth noting that in the Bayesian filtering of the embodiment of the present invention, the state information and control input of the construction robot are represented separately, which can more accurately model and predict the dynamic behavior of the construction robot; and the state information such as the position and speed of the construction robot are integrated in the same state vector. This is because the position and speed jointly describe the dynamic state of the robot at the current moment, reflecting the specific position of the robot in the target space and its motion state (such as speed, acceleration, etc.), and the posterior state estimation Usually includes all variables that can describe the current state of the robot, namely: in, represents the posterior estimate of the position coordinates of the construction robot at the previous moment, represents the posterior estimate of the speed of the construction robot at the previous moment, Represents the posterior estimate of the orientation angle of the construction robot at the previous moment.

[0039] Combining the above four variables in one state vector helps to uniformly process and update the overall state of the robot in the prediction and update steps of the Bayesian filter.

[0040] Specifically, the update steps include: The environmental data is integrated with the prior state estimate to update the estimate of the environmental state. The Kalman gain is calculated by obtaining the environmental data at the current moment. The calculation expression of the Kalman gain is: The posterior state estimate is updated based on the Kalman gain, and the update expression of the posterior state estimate is: Based on the Kalman gain and the observation matrix, the posterior covariance matrix is ​​updated, and the expression is: in, Represents the posterior state estimate, including the new orientation angle , represents the prior state estimate, that is, the state predicted by the model, represents the Kalman gain, Indicates time The observed value of represents the observation matrix, express The transposed matrix of represents the observation noise covariance matrix, represents the identity matrix, represents the posterior state estimate covariance matrix, represents the prior state covariance matrix.

[0041] Specifically, the posterior state estimate is mapped to a two-dimensional grid map to obtain an occupancy grid map. The process is as follows: Ensure that all environmental data is processed within a unified reference framework; The continuous environment space is divided into a two-dimensional grid of fixed size, and each grid cell represents a small area in the environment; For each grid cell, the probability value of the cell being occupied is calculated by analyzing the point cloud density and distance information in the corresponding area in the environmental data. Bayesian updating is usually used to combine the environmental data with the prior state estimate to dynamically adjust the occupancy state of each grid cell. In order to improve the accuracy and reliability of the map, the Kalman filter technique is used to further process the occupancy probability of each grid cell and remove misjudgments caused by sensor noise; In addition, combined with time series data, a dynamic update mechanism is used to reflect moving obstacles and environmental changes in real time, ensuring that the occupancy grid map always accurately reflects the current environmental status; Finally, a two-dimensional occupancy grid map of size 200×200 is generated, which provides a reliable environment representation basis for subsequent path planning and obstacle avoidance decisions.

[0042] In an embodiment of the present invention, the size of the occupied grid map is determined according to the working area of ​​the construction robot, the resolution is set to 0.05 meters, and each grid stores information on whether the corresponding position is occupied by an obstacle.

[0043] Most preferably, the trained lightweight convolutional neural network comprises: The input layer, the first convolutional layer, the first pooling layer, the second convolutional layer, the second pooling layer, the fully connected layer and the output layer are connected in sequence; The input layer is used to receive the occupancy grid map; The first convolutional layer and the second convolutional layer are both used to extract features from the occupancy grid map; The first pooling layer and the second pooling layer are both used to downsample the extracted features; The fully connected layer is used to flatten the features output by the second pooling layer to obtain the environment feature vector; The output layer generates environmental feature data based on the environmental feature vector.

[0044] In an embodiment of the present invention, the first convolution layer uses 32 3×3 convolution kernels, the step size is 1, and the activation function is the ReLU function. The second convolution layer uses 64 3×3 convolution kernels, the step size is 1, and the activation function is the ReLU function. The first pooling layer and the second pooling layer are both maximum pooling layers, the window size is 2×2, and the step size is also 2. The fully connected layer is used to flatten the features output by the convolution layer and output a feature vector of a fixed length.

[0045] Convolutional neural networks are pre-trained to optimize parameters and can efficiently extract the spatial features of the environment.

[0046] Further, the trained decision model includes a policy network and a value network; The policy network generates action probability distribution based on the environmental feature data and the current state of the target construction robot; The value network generates state value estimates based on the environmental feature data and the current state of the target construction robot.

[0047] In the embodiment of the present invention, the PPO algorithm is used to train the decision model. The input of the decision model is the environmental feature data and the current state of the construction robot, and the output is the action probability distribution. The functions of the action probability distribution are: ①Define strategy and action selection: In reinforcement learning, the strategy Defines that in a given state Take action The policy network outputs this probability distribution, which clarifies the possibility of selecting each action in each state; The robot samples an action from the output action probability distribution and executes it. This method introduces randomness, so that the robot may take different actions in the same state, which helps to explore the environment and avoid falling into the local optimal solution; ②Balance exploration and exploitation: Exploring new strategies: Through probability distribution, the strategy network can try multiple action combinations during training and explore new strategy spaces; ③Strategy update and optimization: Calculate policy gradient: In the PPO algorithm, the calculation of policy gradient depends on the policy probability ratio; Calculation of advantage function: The action probability distribution is closely related to the calculation of the advantage function, which measures how advantageous it is to take an action in a state relative to the average policy; Guided policy improvement: By calculating the advantage function, the policy network is able to identify which actions are more advantageous under the current policy, thereby tending to increase the probability of selecting these actions during training.

[0048] Therefore, this action probability distribution defines the probability of various possible actions being selected under given environmental features and robot states, and guides the robot on how to sample and execute actions. By outputting probability distribution, the policy network introduces randomness, so that the robot may take different actions in the same state, thereby promoting the exploration of the environment and avoiding the strategy from falling into the local optimum; at the same time, the action probability distribution is the basis of policy gradient calculation, which is used to calculate the policy probability ratio and construct the clipping loss function in policy optimization to ensure that the amplitude of policy update is limited and maintain the stability of training; in addition, the action probability distribution allows the robot to flexibly adjust action selection in dynamic and complex environments, improve adaptability and robustness, and especially in continuous action space, it can achieve fine action control. In short, the action probability distribution is not only used to define strategies and guide action selection in the policy network, but also plays an important role in policy optimization, exploring the environment, and adapting to complex task requirements. It is the key to achieving efficient obstacle avoidance path planning.

[0049] Specifically, the PPO (Proximal Policy Optimization) algorithm is a reinforcement learning algorithm that solves the problem of PolicyGradient step-size sensitivity. Before training the decision model based on the PPO algorithm, it also includes: Building a reinforcement learning environment Build a simulation environment similar to the actual construction site in simulation software (such as Gazebo), including dynamic obstacles (such as moving personnel models) and static obstacles (such as stacked material models); Designing the reward function The reward function is: in, represents the total reward, represents the time efficiency reward, represents goal-oriented reward, represents obstacle avoidance penalty; Since safety (obstacle avoidance) takes precedence over efficiency (time) and goal orientation in the construction robot path planning, the weight of the obstacle avoidance penalty is set to a high value to ensure that the robot prioritizes avoiding collisions. According to the rule of thumb, the initial setting weight coefficient is , , .

[0050] The specific definitions of each reward part are as follows: Time efficiency bonus: ; Indicates the time increment consumed by the robot in the current decision. This reward is directly related to time consumption and encourages the robot to reduce time consumption and complete the task as quickly as possible. Goal-Oriented Rewards: ; Indicates the Euclidean distance between the robot's current position and the target position. This reward encourages the robot to get closer to the target position. The closer the distance, the smaller the reward (the absolute value of the negative value). Obstacle Avoidance Penalty: ; Represents the collision penalty constant, which takes a positive value. If the robot collides, a fixed negative reward is given to punish dangerous behaviors and ensure safety.

[0051] The training process of the decision model based on the PPO algorithm in the embodiment of the present invention is as follows: The policy network is trained. The input of the policy network is the environmental feature data and the current state of the construction robot. The output is the action probability distribution, including the forward speed. and steering angle ; Use the simulation environment to collect a large number of states, actions, and reward sequences, use the clipping loss function of the PPO algorithm, update the parameters of the policy network and the value network, and adjust the learning rate, batch size and other hyperparameters during the training process to ensure the stability and convergence speed of the model. Specifically: Sampling: In a simulated environment, a large number of state, action, and reward sequences are collected according to the current strategy; Using the sampled sequence data, calculate the advantage function estimate for each time step , providing a basis for updating the policy network, and using the generalized advantage estimation (GAE) method to calculate the advantage function estimate .

[0052] First, calculate the return, starting from time Starting discount cumulative reward (reward) , calculated as: ; in, is the discount factor, , For the moment Instant rewards, The time span for reward calculation, usually until the end of the round or a set maximum number of steps, For the value network state Estimated value of; Second, perform value estimation using the value network For each state Perform value estimation. The value network is similar to the policy network in structure, but is used to estimate the value of the state rather than the action probability distribution. Finally, the advantage function is estimated, and the calculation formula is: This estimate represents the The difference between the actual reward and the value network prediction reflects the effect of taking action degree of advantage.

[0053] Policy Update: Using the calculated advantage function estimate and the clipping loss function of the PPO algorithm, the parameters of the policy network and the value network are updated to improve the performance of the strategy and the stability of the model.

[0054] Policy Network Updates: First calculate the strategy probability ratio: ; in, For the parameter Under the strategy, the robot is in state Take action The probability of , which represents the probability distribution of the action selected by the current strategy, For the parameter Under the strategy, the robot is in state Take action The probability of , which represents the probability distribution of the action selected by the current strategy, For the robot at time step The actions taken, including the robot's forward speed and steering angle , For robots in time In the embodiment of the present invention, the environmental feature refers to the environmental feature representation extracted by Bayesian filtering and lightweight CNN of sensor data, reflecting the location and distribution of obstacles in the current environment.

[0055] Second, construct the clipping loss function: in, are policy network parameters, is the advantage function estimate, representing the action Advantages relative to the average; is the clipping function, which limits the amplitude of the strategy update. is the shearing range, usually 0.1 or 0.2, Represents the time step Expected value on Finally, update the value network parameters: use gradient descent or other optimization algorithms to minimize the loss function , update the parameters of the value network ; is the learning rate of the policy network.

[0056] Value Network Update: The training process of the value network is carried out synchronously with the policy network, aiming to improve the value estimation of each state in the environment, thereby supporting the optimization of the policy network; specifically, the training process of the value network includes the following steps: First, we construct the loss function of the value network. We use the mean squared error (MSE) as the loss function of the value network. With actual return The difference between: Among them, the loss function is the mean square error (MSE), which measures the value network prediction With actual return The difference between.

[0057] is the batch size, i.e. the number of data samples used for one parameter update; Second, update the value network parameters and use gradient descent or other optimization algorithms to minimize the loss function. , update the parameters of the value network : is the learning rate of the value network.

[0058] Train the policy network and value network simultaneously: The updates of the policy network and the value network are usually performed synchronously on the same batch of data to ensure the accuracy of the advantage function estimation and the consistency of the policy update.

[0059] Hyperparameter Tuning: Adjust the learning rate appropriately according to the training results , , Discount Factor , Cutting parameters Adjust the batch size N and the number of training rounds to balance the training speed and model performance.

[0060] Repeat the iteration: The three steps of sampling, advantage function calculation, and policy updating are repeated until the performance of the policy network and the value network meet the preset requirements, such as converging to a stable reward value or achieving the expected task success rate.

[0061] The embodiment of the present invention obtains environmental data of the target space; fuses the state information of the target construction robot with the environmental data through Bayesian filtering to obtain an occupied grid map; inputs the occupied grid map into a trained lightweight convolutional neural network for feature extraction to generate environmental feature data; inputs the environmental feature data and the current state of the target construction robot into a trained decision model for decision-making to obtain action instructions, and the target construction robot reaches the target position in the shortest time based on the action instructions, and the path taken by the target construction robot to the target position is the obstacle avoidance path of the target construction robot; compared with the prior art, the embodiment of the present invention first uses Bayesian filtering to fuse the state information of the target construction robot with the environmental data, thereby improving the accuracy and robustness of environmental perception, and uses a lightweight convolutional neural network and an optimized reinforcement learning network to extract data features and generate action instructions, thereby reducing computational complexity, improving decision-making speed, and meeting real-time requirements, and finally realizes efficient obstacle avoidance and path planning in complex environments based on action instructions.

[0062] Corresponding to the obstacle avoidance path planning method of the construction robot described in the above embodiment, as Figure 2 As shown, the present invention also provides an obstacle avoidance path planning device 100 for a construction robot, and the obstacle avoidance path planning device 100 includes: An acquisition module 101 is used to acquire environmental data of a target space; A fusion module 102 is used to fuse the state information of the target construction robot with the environmental data through Bayesian filtering to obtain an occupancy grid map; An extraction module 103 is used to input the occupancy grid map into a trained lightweight convolutional neural network for feature extraction to generate environmental feature data; The decision module 104 is used to input the environmental feature data and the current state of the target construction robot into the trained decision model to make a decision and obtain an action instruction. The target construction robot reaches the target position in the shortest time based on the action instruction. The path taken by the target construction robot to the target position is the obstacle avoidance path of the target construction robot.

[0063] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0064] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0065] The present invention also provides a terminal device, such as Figure 3 As shown, the terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 3 Only one processor is shown in the figure), a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100. When the processor D100 executes the computer program D102, the above-mentioned obstacle avoidance path planning method for the construction robot is implemented.

[0066] The terminal device D10 may be a computing device such as a desktop computer, a notebook, a PDA, a server, a server cluster, a cloud server, etc. The terminal device may include, but is not limited to, a processor D100 and a memory D101. Those skilled in the art will appreciate that Figure 3 This is only an example of the terminal device D10 and does not constitute a limitation on the terminal device D10. The terminal device D10 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0067] The processor D100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0068] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may also be an external storage device of the terminal device D10, such as a plug-in hard disk, a smart memory card (SMC, SmartMedia Card), a secure digital (SD, Secure Digital) card, a flash card (Flash Card), etc. equipped on the terminal device D10. Further, the memory D101 may also include both an internal storage unit of the terminal device D10 and an external storage device. The memory D101 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory D101 may also be used to temporarily store data that has been output or is to be output.

[0069] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0070] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0071] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the obstacle avoidance path planning method of the construction robot is implemented.

[0072] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the construction device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0073] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for obstacle avoidance path planning of a construction robot, characterized in that: include: Step 1, obtaining the environmental data of the target space; Step 2, fusing the state information of the target construction robot with the environmental data through Bayesian filtering to obtain an occupancy grid map; Step 3, inputting the occupancy grid map into the trained lightweight convolutional neural network for feature extraction to generate environmental feature data; Step 4: Input the environmental feature data and the current state of the target construction robot into the trained decision model to make a decision and obtain an action instruction. The target construction robot reaches the target position in the shortest time based on the action instruction. The path taken by the target construction robot to the target position is the obstacle avoidance path of the target construction robot.

2. The obstacle avoidance path planning method for a construction robot according to claim 1, characterized in that: The step 1 comprises: Setting up a laser radar and multiple ultrasonic sensors inside the target construction robot; Acquire three-dimensional point cloud data of the target space through the laser radar; Acquiring distance data between the target construction robot and obstacles through a plurality of ultrasonic sensors; The three-dimensional point cloud data and the distance data are integrated into the environment data of the target space.

3. The obstacle avoidance path planning method for a construction robot according to claim 2, characterized in that: Before fusing the state information of the target construction robot with the environmental data through Bayesian filtering, the method further includes: Preprocessing the environmental data to obtain preprocessed environmental data; The state information of the target construction robot is fused with the preprocessed environmental data through Bayesian filtering data.

4. The obstacle avoidance path planning method for a construction robot according to claim 3, characterized in that: The step 2 comprises: Obtaining the position, movement speed and action instruction of the target construction robot at the last moment; Based on the position, movement speed and action instruction of the target construction robot at the previous moment, a priori state estimation is generated using a kinematic model, wherein the priori state estimation includes the current position, speed and steering information of the target construction robot; fusing the preprocessed environmental data with the priori state estimate to obtain a posterior state estimate; The posterior state estimate is mapped onto a two-dimensional grid map to obtain an occupancy grid map.

5. The obstacle avoidance path planning method for a construction robot according to claim 4, characterized in that: The calculation expression of the posterior state estimation is: in, represents the posterior state estimate, represents the prior state estimate, represents the Kalman gain, Indicates time The observed value of represents the observation matrix, express The transposed matrix of represents the observation noise covariance matrix, represents the identity matrix, represents the posterior state estimate covariance matrix, represents the prior state covariance matrix.

6. The obstacle avoidance path planning method for a construction robot according to claim 5, characterized in that: The trained lightweight convolutional neural network includes: The input layer, the first convolutional layer, the first pooling layer, the second convolutional layer, the second pooling layer, the fully connected layer and the output layer are connected in sequence; The input layer is used to receive the occupancy grid map; The first convolutional layer and the second convolutional layer are both used to extract features from the occupancy grid map; The first pooling layer and the second pooling layer are both used to downsample the extracted features; The fully connected layer is used to flatten the features output by the second pooling layer to obtain an environment feature vector; The output layer generates environmental feature data based on the environmental feature vector.

7. The obstacle avoidance path planning method for a construction robot according to claim 6, characterized in that: The trained decision model includes a policy network and a value network; The policy network generates an action probability distribution based on the environmental feature data and the current state of the target construction robot; The value network generates a state value estimate based on the environmental characteristic data and a current state of the target construction robot.

8. An obstacle avoidance path planning device for a construction robot, characterized in that: include: An acquisition module is used to obtain the environmental data of the target space; A fusion module, used to fuse the state information of the target construction robot with the environmental data through Bayesian filtering to obtain an occupancy grid map; An extraction module, used for inputting the occupancy grid map into a trained lightweight convolutional neural network for feature extraction to generate environmental feature data; A decision-making module is used to input the environmental feature data and the current state of the target construction robot into a trained decision model to make a decision and obtain an action instruction. The target construction robot reaches the target position in the shortest time based on the action instruction, and the path taken by the target construction robot to the target position is the obstacle avoidance path of the target construction robot.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the obstacle avoidance path planning method for the construction robot as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the obstacle avoidance path planning method for a construction robot as described in any one of claims 1 to 7 is implemented.

Images