Mobile robot path planning method based on deep reinforcement learning

Abstract

The invention discloses a mobile robot path planning method based on deep reinforcement learning, and relates to the technical field of robot path planning. According to the method, a mobile robot path is divided into global planning and local obstacle avoidance, and local obstacle avoidance is realized through a BiLSTM-D3QN model; analyzing the first sensing data through a decision branch of the BiLSTM model to obtain a decision parameter, and comparing the decision parameter with a set parameter threshold to judge whether local obstacle avoidance is performed or not; when it is determined that local obstacle avoidance needs to be carried out, the BiLSTM model outputs obstacle avoidance parameters, then a D3QN model is used for generating a local obstacle avoidance path, and autonomous obstacle avoidance of the mobile robot is achieved; according to the invention, the BiLSTM model bidirectionally captures the context dependency relationship of the time sequence data, can simultaneously fuse forward and backward time sequence features, judges whether local obstacle avoidance is needed or not by using multi-dimensional data, defines the access opportunity of local obstacle avoidance, and improves the accuracy and reliability of local obstacle avoidance.

Description

Technical Field

[0001] The present invention belongs to the technical field of robot path planning, and specifically to a mobile robot path planning method based on deep reinforcement learning. Background Art

[0002] Path planning is a core issue in the field of mobile robotics. It can be categorized into global path planning, where environmental information is known, and local path planning, where environmental information is partially known or unknown. Traditional navigation algorithms, such as the A* algorithm and the Dijkstra algorithm, rely heavily on environmental maps for path planning. However, in dynamic, open environments, mobile robots lack access to environmental maps in certain scenarios, such as fire rescue and earthquake relief. This results in inefficient path planning. The current trend toward intelligent and autonomous robots requires that they possess the ability to learn and adapt to changing environments.

[0003] The present invention provides a mobile robot path planning method based on deep reinforcement learning to solve the above technical problems. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a mobile robot path planning method based on deep reinforcement learning, which is used to solve the technical problem of low path planning efficiency in the prior art in scenarios with environmental information location.

[0005] To achieve the above objectives, a first aspect of the present invention provides a mobile robot path planning method based on deep reinforcement learning, comprising:

[0006] Planning the global path of the mobile robot based on the work task;

[0007] Acquire first sensor data in real time; analyze the first sensor data through the decision branch of the BiLSTM model to determine whether to perform local obstacle avoidance; if so, output obstacle avoidance parameters through the parameter branch of the BiLSTM model; wherein the obstacle avoidance parameters include a steering angle and a speed correction;

[0008] Acquire second sensor data; analyze the second sensor data using a D3QN model to generate a local obstacle avoidance path; wherein the second sensor data includes the first sensor data and obstacle avoidance parameters;

[0009] Under the guidance of the global path, autonomous obstacle avoidance is achieved based on the local obstacle avoidance path.

[0010] Preferably, the first sensing data includes environmental perception characteristics and mobile robot state characteristics; the second sensing includes the first sensing data, obstacle avoidance parameters, task target data and priority parameters.

[0011] Preferably, determining whether to perform local obstacle avoidance includes:

[0012] Extract the decision parameters and corresponding parameter thresholds output by the decision branch;

[0013] When the decision parameter is greater than the parameter threshold, it is determined that local obstacle avoidance is required; otherwise, it is determined that local obstacle avoidance is not required.

[0014] Preferably, when the decision branch determines that the local obstacle avoidance condition is not met, the parameter branch does not generate the obstacle avoidance parameter.

[0015] Preferably, when the decision branch determines that the local obstacle avoidance condition is not met, the parameter branch does not output the obstacle avoidance parameter.

[0016] Preferably, adaptively adjusting the parameter threshold based on obstacle density includes:

[0017] Extract the base value and the standard density corresponding to the base value; when the obstacle density is the standard density, the local obstacle avoidance capability is optimal when the parameter threshold is set to the base value;

[0018] Obtain the obstacle density, and use the ratio of the obstacle density to the standard density as the density coefficient; correct the basic value based on the density coefficient to obtain the corrected parameter threshold.

[0019] Preferably, the parameter threshold is adaptively adjusted based on the average missed detection rate and the average efficiency loss, including:

[0020] Obtain the average missed detection rate and average efficiency loss within a set period; use the ratio of the average missed detection rate to the standard ratio as the missed detection coefficient, and use the ratio of the average efficiency loss to the standard loss as the loss coefficient;

[0021] The basic value is corrected based on the missed detection coefficient and the loss coefficient to obtain the updated parameter threshold.

[0022] Preferably, the parameter threshold is adaptively adjusted based on the obstacle density, the average missed detection rate, and the average efficiency loss, including:

[0023] Extract the base value and the standard ratio, standard loss, and standard density corresponding to the base value. The standard ratio and standard loss are the average missed detection rate and average efficiency loss corresponding to the base value obtained during the test. When the obstacle density is the standard density, the local obstacle avoidance capability is optimal when the parameter threshold is set to the base value.

[0024] Obtain the average missed detection rate and average efficiency loss within the set period, as well as the current obstacle density; use the ratio of the average missed detection rate to the standard ratio as the missed detection coefficient, the ratio of the average efficiency loss to the standard loss as the loss coefficient; and the ratio of the obstacle density to the standard density as the density coefficient;

[0025] The basic value is corrected based on the density coefficient, missed detection coefficient and loss coefficient to obtain the updated parameter threshold.

[0026] Preferably, obtaining the priority coefficient includes:

[0027] Extract the shortest collision time between the mobile robot and the obstacle and the remaining time of the work task;

[0028] The inverse of the product of the shortest collision time and the remaining time is mapped into a standard range, and the mapping result is used as the priority parameter; wherein the standard range is [0, 1].

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. In the present invention, the mobile robot path is divided into global planning and local obstacle avoidance, and local obstacle avoidance is implemented through the BiLSTM-D3QN model; the decision branch of the BiLSTM model analyzes the first sensor data to obtain a decision parameter, and the decision parameter is compared with the set parameter threshold to determine whether local obstacle avoidance is to be performed; when it is determined that local obstacle avoidance is required, the BiLSTM model outputs the obstacle avoidance parameter, and then uses the D3QN model to generate a local obstacle avoidance path to achieve autonomous obstacle avoidance of the mobile robot; the BiLSTM model in the present invention can simultaneously fuse forward and backward time series features by bidirectionally capturing the contextual dependency of time series data, use multi-dimensional data to determine whether local obstacle avoidance is required, clarify the access timing of local obstacle avoidance, and then automatically generate a local obstacle avoidance path, which can improve the accuracy and reliability of local obstacle avoidance.

[0031] 2. The present invention determines whether to perform local obstacle avoidance by comparing the decision parameters with the set parameter thresholds. In setting the parameter thresholds, the ratio of the obstacle density to the standard density can be used to correct the base value, thereby achieving timely adjustment of the parameter threshold according to the complexity of the environment. The parameter threshold can also be adjusted in an inverse manner using the average missed detection rate and the average efficiency loss within a set period. Of course, the obstacle density can also be combined with the average missed detection rate and the average efficiency loss within a set period to achieve adaptive adjustment of the parameter threshold. The present invention adaptively adjusts the parameter threshold through multi-dimensional data, which can accurately determine whether local obstacle avoidance is needed in different scenarios, and can also improve the access accuracy of local obstacle avoidance for emergencies. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0033] Figure 1 Schematic diagram of the method steps of the mobile robot path planning method based on deep reinforcement learning in the present invention;

[0034] Figure 2 A schematic diagram of the steps of determining whether to perform local obstacle avoidance in the present invention;

[0035] Figure 3 Schematic diagram of the method for adaptively adjusting parameter thresholds in the present invention Figure 1 ;

[0036] Figure 4 Schematic diagram of the method for adaptively adjusting parameter thresholds in the present invention Figure 2 ;

[0037] Figure 5 Schematic diagram of the method for adaptively adjusting parameter thresholds in the present invention Figure 3 . DETAILED DESCRIPTION

[0038] The technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0039] See also Figure 1 The first embodiment of the present invention provides a mobile robot path planning method based on deep reinforcement learning, comprising:

[0040] S100: Planning the global path of the mobile robot based on the work task;

[0041] S200: Acquire first sensor data in real time; analyze the first sensor data through the decision branch of the BiLSTM model to determine whether to perform local obstacle avoidance; if yes, output obstacle avoidance parameters through the parameter branch of the BiLSTM model;

[0042] S300: Acquire second sensor data; analyze the second sensor data using a D3QN model to generate a local obstacle avoidance path; wherein the second sensor data includes the first sensor data and obstacle avoidance parameters;

[0043] S400: Under the guidance of the global path, autonomous obstacle avoidance is achieved based on the local obstacle avoidance path.

[0044] S100: Planning the global path of the mobile robot based on the work task.

[0045] As an example and not a limitation, the global path of a mobile robot is planned based on the A* algorithm:

[0046] First, obtain the task map corresponding to the work task, rasterize the task map, and use the grid assignment method to distinguish whether the grid belongs to the passable area of ​​the mobile robot. Specifically, the passable grid can be assigned a value of 0 and other grids can be assigned a value of 1.

[0047] Next, the task point corresponding to the work task and the grid coordinates corresponding to the task point are determined. Based on this data, the A* algorithm is used to perform a path search and plan the global path for the mobile robot to perform the work task. The specific steps of the A* algorithm for path planning are well known and will not be detailed here.

[0048] S200: Acquire first sensor data in real time; analyze the first sensor data through the decision branch of the BiLSTM model to determine whether to perform local obstacle avoidance; if yes, output obstacle avoidance parameters through the parameter branch of the BiLSTM model.

[0049] The BiLSTM model is an improved version of the long short-term memory (LSTM) network. It can simultaneously integrate past (forward) and future (backward) time series features by capturing the contextual dependencies of time series data in both directions, providing a more comprehensive understanding of the semantics and structure of time series data. This can address the limitations of recurrent neural networks and unidirectional LSTM networks, which can only use past information (forward sequence) and result in low prediction accuracy.

[0050] Specifically, the unidirectional LSTM algorithm can only utilize historical information from the past, such as using the historical movement trajectory of a mobile robot to predict the movement trend of an obstacle. Since it mainly relies on historical data, it will lead to a lag in local obstacle avoidance decisions. Moreover, for obstacles with non-uniform motion, such as pedestrians that suddenly change direction, and vehicles that accelerate or decelerate, the prediction error is large, and may even lead to the failure of local obstacle avoidance.

[0051] The BiLSTM model uses two LSTM layers, forward and backward, to process time series data from the past-to-present and future-to-present directions, respectively. This approach can identify the movement trends of future obstacles and accurately identify non-uniform-speed obstacles. The local obstacle avoidance path constructed based on this approach is more consistent with actual conditions and can also ensure the reliability of the local obstacle avoidance path.

[0052] The first sensing data includes environmental perception features and mobile robot state features.

[0053] Environmental perception features primarily include obstacle distance, angle, relative speed, size, and category (dynamic or static); they can also include collision time, obstacle density, and more. Obstacle distance, angle, size, and category can be acquired using a LiDAR installed on a mobile robot. For example, LiDAR can be used to detect and obtain high-precision 3D point cloud data, from which distance and angle can be directly extracted. Obstacle categories can then be identified using multiple frames of data. Relative speed can be directly measured using millimeter-wave radar. The collision time is the ratio of obstacle distance to relative speed, and obstacle density can be expressed as the point cloud density of obstacles per unit volume.

[0054] It should be understood that the sensor used to collect the primary sensor data, such as a lidar or millimeter-wave radar, can be installed on the mobile robot, and an auxiliary device (such as a rotation device) can be used to control the sensor's measurement direction. However, it is necessary to ensure that the sensor can cover the data required for global and local obstacle avoidance. Coverage strategies can adopt field of view overlap strategies and redundant designs.

[0055] Mobile robot state characteristics primarily include linear velocity, angular velocity, and steering angle; they may also include global path progress, waypoint deviation, and path curvature. Linear velocity can be obtained using wheel encoders installed on the mobile robot's drive wheels, while angular velocity can be obtained using an inertial measurement unit (IMU) mounted at the robot's center of gravity. Steering angle is obtained using a steering joint encoder installed on the steering shaft or servo output shaft. Global path progress and waypoint deviation can be obtained using the mobile robot's positioning system. Path curvature can be calculated after global path planning is complete.

[0056] After preprocessing, the first sensor data is fed into the trained BiLSTM model. The output of the decision branch determines whether local obstacle avoidance is necessary. If necessary, the parameter branch outputs obstacle avoidance parameters, which serve as a set of input features for the D3QN model to generate a local obstacle avoidance path.

[0057] Preprocessing primarily includes missing value handling, outlier filtering, time synchronization, spatial transformation, and feature extraction. Time synchronization is achieved by aligning timestamps and using linear interpolation to unify the sampling frequency. Spatial transformation converts obstacle coordinates from sensor coordinates to robot coordinates, and then to global map coordinates. During feature extraction, numerical parameters require normalization, and categorical parameters require one-hot encoding.

[0058] The training process of the BiLSTM model is briefly described as follows:

[0059] Collecting operational sensor data of the mobile robot in various scenarios; the content of the operational sensor data is consistent with the content type of the first sensor data;

[0060] The running sensor data is marked. If local obstacle avoidance is required, it is marked as 1, and the optimal obstacle avoidance parameters are also marked.

[0061] The BiLSTM model constructed by running sensor data and corresponding annotation training is stored.

[0062] The network structure of the BiLSTM model mainly includes:

[0063] Input layer: Its dimension is consistent with the number of features of the first sensor data, such as 10 dimensions of environmental perception features + 5 dimensions of robot state features = 15 dimensions;

[0064] Bidirectional LSTM layer: The number of hidden units is determined through experimental tuning and is generally set to 64, 128, 256, etc. It can be set to 128 each for the forward and reverse directions to capture the dependencies between the data. The output of this layer is the combined forward and reverse outputs, resulting in a feature vector containing contextual information.

[0065] Decision branch: includes a fully connected layer and an activation function called ReLU. The output layer is a neuron that outputs a probability value between 0 and 1 through a Sigmoid activation function.

[0066] Parameter branch: includes a fully connected layer and ReLU activation function; the output layer has the same dimension as the obstacle avoidance parameters and is based on linear activation output.

[0067] The BiLSTM model has two output branches: a decision branch and a parameter branch. The decision branch and the parameter branch share the first sensor data.

[0068] The decision branch is used to output a decision parameter corresponding to the input first sensor data, and the decision parameter is used to determine whether local obstacle avoidance is required. Figure 2 , the solution is as follows:

[0069] S211: extracting the decision parameters output by the decision branch and the corresponding parameter thresholds;

[0070] S212: When the decision parameter is greater than the parameter threshold, it is determined that local obstacle avoidance is required; otherwise, it is determined that local obstacle avoidance is not required.

[0071] The decision parameter can be expressed as a probability, with a value range of [0, 1]. The larger the decision parameter, the more necessary local obstacle avoidance is. Generally, the need for local obstacle avoidance is determined by comparing the decision parameter with a set threshold. For example, if the threshold is 0.5, then local obstacle avoidance is considered necessary when the output decision parameter is greater than or equal to 0.5.

[0072] The parameter branch outputs obstacle avoidance parameters corresponding to the first sensor data input. These parameters are a crucial component of the D3QN model's input data and form the basis for the D3QN model's generation of local obstacle avoidance paths. For example, these parameters may include steering angles and speed corrections, which represent the optimal obstacle avoidance parameters for the mobile robot under the given operating conditions.

[0073] Based on S200, the output result of the decision branch of the BiLSTM model determines whether the parameter branch needs to be output. If the decision parameter output by the decision branch is less than the set parameter threshold, it is determined that local obstacle avoidance is not required, and naturally the obstacle avoidance parameter is not required.

[0074] As an optional embodiment of the present application, when the decision branch determines that the local obstacle avoidance condition is not met, the parameter branch does not generate the obstacle avoidance parameter.

[0075] When local obstacle avoidance isn't necessary, obstacle avoidance parameters are not generated. Specifically, the BiLSTM model first determines whether local obstacle avoidance is necessary based on the input first sensor data. If so, the BiLSTM model outputs obstacle avoidance parameters based on the input first sensor data. Because the outputs of the decision branch and parameter branch are sequential, resource demand is relatively low at any one time, alleviating resource pressure.

[0076] As an optional embodiment of the present application, when the decision branch determines that the local obstacle avoidance condition is not met, the parameter branch does not output the obstacle avoidance parameter.

[0077] Obstacle avoidance parameters don't need to be output after they're generated. The BiLSTM model uses the decision branch to generate decision parameters based on the first sensor data input. Then, using the parameter branch, it generates obstacle avoidance parameters. It first determines whether the decision parameters exceed the set parameter threshold. If so, the obstacle avoidance parameters are output; otherwise, the obstacle avoidance parameters are not output. Because the decision branch and parameter branch run in parallel, the obstacle avoidance parameters can be input into the D3QN model immediately if local obstacle avoidance is required. This improves the efficiency of generating local obstacle avoidance paths, thereby controlling the mobile robot to complete local obstacle avoidance in a timely manner.

[0078] S300: Acquire second sensor data; analyze the second sensor data through the D3QN model to generate a local obstacle avoidance path; wherein the second sensor data includes the first sensor data and obstacle avoidance parameters.

[0079] The D3QN (Dueling Double DQN) algorithm combines the decision-making capabilities of reinforcement learning with the feature processing advantages of deep neural networks to achieve real-time obstacle avoidance and path optimization in dynamic environments. The D3QN model's secondary sensory input primarily includes primary sensory data (environmental perception features and mobile robot state characteristics), mission objective data, priority parameters, and obstacle avoidance parameters output by the BiLSTM model.

[0080] Environmental perception features mainly include obstacle information, such as obstacle distance, angle, relative speed, acceleration, size and category, and spatial features, such as local map rasterization and traversable area boundaries.

[0081] The state characteristics of the mobile robot are the same as those in the first sensor data; of course, some additional auxiliary characteristics may also be added, such as the remaining power of the mobile robot.

[0082] The mission target data mainly includes the global path point sequence, the current target point coordinates, the mission priority and deadline, etc.

[0083] The D3QN model training process is not detailed here. The input data preprocessing process can be referenced for the preprocessing of the first sensor data. The D3QN model inputs the second sensor data, and its output is the Q-value estimate for discrete actions (e.g., turn left, turn right, accelerate, decelerate, hold).

[0084] The network structure of the D3QN model mainly includes:

[0085] Input layer: Its dimension is consistent with the dimension of the second sensor data, for example, the first sensor data is 15 dimensions + the obstacle avoidance parameters are 2 dimensions + the task objectives are 3 dimensions + the priority is 1 dimension = 21 dimensions;

[0086] Hidden layer: fully connected layer 1, number of neurons 256, activation function ReLU; fully connected layer 2, number of neurons 128, activation function ReLU;

[0087] Dueling network branches: Value stream, outputting a scalar value V(s), representing the value of the current state; Advantage stream, outputting a vector A(a|s), representing the advantage of each action (action space dimension = 5, e.g., 5 discrete actions);

[0088] Q value calculation:

[0089] The priority parameter refers to the priority of local obstacle avoidance, and its purpose is to improve the rationality and safety of the mobile robot's decision-making in complex environments. The priority parameter is calculated as follows:

[0090] S311: extracting the shortest collision time between the mobile robot and the obstacle and the remaining time of the work task;

[0091] S312: Map the inverse of the product of the shortest collision time and the remaining time into a standard range, and use the mapping result as a priority parameter; wherein the standard range is [0, 1].

[0092] The shortest collision time is the minimum time between the mobile robot and each obstacle, if the mobile robot and the obstacle remain in their current state. The priority parameter can be mapped to the standard range using a piecewise function mapping method. The specific method of piecewise function mapping is as follows:

[0093] The priority parameters are: Among them, TTC is the shortest collision time, T remain The remaining time for the work task;

[0094] The normalization formula of the piecewise function is: Among them, T threshold is the task urgency threshold (e.g. 60 seconds), TTC safe is the safety collision time threshold (such as 2 seconds), is the scaling factor, which controls the steepness of the curve. You can take T threshold ×TTC safe .

[0095] In the normalization formula, if P′ tends to 0, the normalized result P tends to 0; if P′ tends to ∞, the normalized result P tends to 1.

[0096] S400: Under the guidance of the global path, autonomous obstacle avoidance is achieved based on the local obstacle avoidance path.

[0097] The mobile robot moves along the global path. If local obstacle avoidance is required, a local obstacle avoidance path will be generated. At this time, the path fusion algorithm can be used to transition from the global path to the local obstacle avoidance path.

[0098] For path transition, as an example and not a limitation:

[0099] Set the global path weight: Among them, d obs is the current obstacle distance, d sale is the safety distance threshold, k is the weight change coefficient; local obstacle avoidance weight w 1ocal =1-w global ;

[0100] The path point fusion transition formula is: P fused (t) = w global (t)·G(t)+w local(t)·L(t); where G(t) is the target point at time t in the global path, and L(t) is the obstacle avoidance point at time t in the local obstacle avoidance path; P fused (t) is the path point after fusion.

[0101] Embodiment 2: Compared with embodiment 1, this embodiment adaptively adjusts the parameter threshold, which is compared with the decision parameter output by the decision branch to determine whether local obstacle avoidance is required.

[0102] As a preferred embodiment of the present application, the parameter threshold value compared with the decision parameter can be adaptively modified, specifically according to the obstacle density in the time series sensor data. Since the higher the obstacle density, the higher the collision risk of the scene where the mobile robot is located, the parameter threshold value should be lowered to trigger local obstacle avoidance in advance. Figure 3 , the solution is as follows:

[0103] S221: extracting a base value and a standard density corresponding to the base value; wherein, when the obstacle density is the standard density, the local obstacle avoidance capability is optimal when the parameter threshold is set to the base value;

[0104] S222: Obtain obstacle density, and use the ratio of the obstacle density to the standard density as a density coefficient; and correct the basic value based on the density coefficient to obtain a corrected parameter threshold.

[0105] For example, assuming that the basic value of the parameter threshold is 0.5 and the ratio of the obstacle density to the standard density is 0.9, the updated parameter threshold is CY=CJ×MX=0.5×0.9=0.45; where CY is the parameter threshold, CJ is the basic value, and MX is the density coefficient.

[0106] The standard density is set empirically or obtained through simulation. When the obstacle density is at the standard density, the parameter threshold for the mobile robot's optimal local obstacle avoidance capability serves as the base value. Furthermore, the obstacle density is the current obstacle density, i.e., the most recent obstacle density obtained by the mobile robot's sensors.

[0107] For example, in a simulation, a mobile robot is simulated performing local obstacle avoidance within a set obstacle density. The parameter threshold is then modified to analyze the robot's obstacle avoidance effectiveness. The optimal parameter threshold is then associated with the set obstacle density. If this parameter threshold is set as the base value, the standard density becomes the associated obstacle density.

[0108] By adaptively adjusting parameter thresholds based on obstacle density, the mobile robot can perform local obstacle avoidance in high-density obstacle scenarios, thereby improving the success rate of obstacle avoidance in high-risk scenarios. Meanwhile, in low-density obstacle scenarios, the mobile robot can reduce the frequency of local obstacle avoidance and promptly eliminate redundant obstacle avoidance.

[0109] As a preferred embodiment of the present application, the parameter threshold can also be adaptively modified based on the historical data of local obstacle avoidance, specifically, adaptively modified based on the missed detection rate and efficiency loss of local obstacle avoidance. Figure 4 , the solution is as follows:

[0110] S231: extracting a base value and a standard ratio and a standard loss corresponding to the base value; wherein the standard ratio and the standard loss are the average missed detection rate and the average efficiency loss corresponding to the base value obtained during the test;

[0111] S232: Obtain an average missed detection rate and an average efficiency loss within a set period; use the ratio of the average missed detection rate to the standard ratio as a missed detection coefficient, and use the ratio of the average efficiency loss to the standard loss as a loss coefficient;

[0112] S233: Correct the basic value based on the missed detection coefficient and the loss coefficient to obtain an updated parameter threshold.

[0113] The missed detection rate represents the ratio of the number of collisions that occurred during the mobile robot's operation due to the absence of obstacle avoidance triggering to the total number of collisions. A higher ratio indicates a higher parameter threshold setting and should be lowered. The efficiency loss refers to the increase in path time consumed by the mobile robot due to obstacle avoidance. A higher efficiency loss indicates a higher threshold setting.

[0114] For example, assuming that the basic value of the parameter threshold is 0.5, the ratio of the missed detection rate to the standard ratio is 1.2, and the ratio of the efficiency loss to the standard loss is 1; then the updated parameter threshold is CY=CJ×LX×SX=0.5×1.2×1=0.6; where LX is the missed detection coefficient and SX is the loss coefficient.

[0115] The standard ratio and standard loss corresponding to the basic value are set based on experience or obtained through simulation. When the parameter threshold is the basic value, the average missed detection rate and average efficiency loss corresponding to the mobile robot performing local obstacle avoidance are obtained.

[0116] Taking the simulation of obtaining the standard ratio and standard loss as an example, we simulated a mobile robot performing local obstacle avoidance while simultaneously adjusting parameter thresholds. We statistically analyzed the average missed detection rate and average efficiency loss corresponding to each parameter threshold, and then correlated the parameter threshold with the average missed detection rate and average efficiency loss. When the parameter threshold is used as the base threshold, the average missed detection rate is used as the standard ratio, and the average efficiency loss is used as the standard loss.

[0117] In some other preferred embodiments, the parameter threshold can be adaptively modified by combining obstacle density, missed detection rate and efficiency loss. Figure 5 , the solution is as follows:

[0118] S241: Extracting the base value and the standard ratio, standard loss, and standard density corresponding to the base value; wherein the standard ratio and standard loss are the average missed detection rate and average efficiency loss corresponding to the base value obtained during testing; when the obstacle density is the standard density, the local obstacle avoidance capability is optimal when the parameter threshold is set to the base value;

[0119] S242: Obtain the average missed detection rate and average efficiency loss within a set period and the current obstacle density; use the ratio of the average missed detection rate to the standard ratio as the missed detection coefficient, the ratio of the average efficiency loss to the standard loss as the loss coefficient; and the ratio of the obstacle density to the standard density as the density coefficient;

[0120] S243: Correct the basic value based on the density coefficient, the missed detection coefficient, and the loss coefficient to obtain an updated parameter threshold.

[0121] It should be noted that parameter thresholds are adjusted periodically because obstacle density is available in real time, but the average missed detection rate and average efficiency loss require historical statistical data. If baseline values ​​are adjusted based on the density coefficient, missed detection coefficient, and loss coefficient, the parameter thresholds must be updated periodically. This means that the parameter thresholds are updated once every set period, with the parameter thresholds updated in the previous period serving as the baseline for the current period. Furthermore, the baseline values ​​for the current period are updated using the average missed detection rate and average efficiency loss from the previous period.

[0122] For example, assuming that the basic value of the parameter threshold is 0.5, the density coefficient is 0.9, the missed detection coefficient is 1.2, and the loss coefficient is 1, the updated parameter threshold is CY=CJ×MX×LX×SX=0.5×0.9×1.2×1=0.554; where CY is the parameter threshold, CJ is the basic value, MX is the density coefficient, LX is the missed detection coefficient, and SX is the loss coefficient.

[0123] It is worth noting that in the above-mentioned adaptive update scheme of parameter thresholds, there are differences in the understanding of the values ​​of standard data (standard density, standard ratio and standard loss).

[0124] When adjusting the parameter threshold based on obstacle density, the standard density is associated with the base value. This means that when the obstacle density is at the standard density, and other factors are not considered, the mobile robot's local obstacle avoidance capability is optimal when the parameter threshold is set to the base value. Similarly, when other factors are not considered and the parameter threshold is set to the base value, the mobile robot's average missed detection rate should be the standard rate, and its average efficiency loss should be the standard loss.

[0125] However, if the parameter threshold is the base value, but the obstacle density is not equal to the standard density, the base value cannot be used for decision-making in the current environment and needs to be adjusted based on the current obstacle density. Similarly, if the parameter threshold is the base value, the base value needs to be adjusted if the average missed detection rate and average efficiency loss deviate.

[0126] It should also be understood that since the average missed detection rate and average efficiency loss are data within a set period, that is, historical data; if the average missed detection rate and average efficiency loss deviate, it may be because the working scenario of the mobile robot has changed, or it may be because of some sudden access reasons that lead to inaccurate judgment of the local obstacle avoidance access timing. Therefore, adjusting the parameter threshold based on the comprehensive average missed detection rate and average efficiency loss can improve the reliability of local obstacle avoidance.

[0127] The above embodiments are only used to illustrate the technical method of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A mobile robot path planning method based on deep reinforcement learning, characterized in that: include: Planning the global path of the mobile robot based on the work task; Acquiring first sensor data in real time; Analyzing the first sensor data through the decision branch of the BiLSTM model to determine whether to perform local obstacle avoidance; If yes, the parameter branch of the BiLSTM model outputs obstacle avoidance parameters, which include steering angle and speed correction. Acquire second sensor data; and generate a local obstacle avoidance path by analyzing the second sensor data through a D3QN model; wherein the second sensor data includes the first sensor data and obstacle avoidance parameters; Under the guidance of the global path, autonomous obstacle avoidance is achieved based on the local obstacle avoidance path.

2. The mobile robot path planning method based on deep reinforcement learning according to claim 1 is characterized in that: The first sensing data includes environmental perception characteristics and mobile robot state characteristics; the second sensing includes the first sensing data, obstacle avoidance parameters, task target data and priority parameters.

3. The mobile robot path planning method based on deep reinforcement learning according to claim 1, characterized in that: Determine whether to perform local obstacle avoidance, including: Extracting the decision parameters and corresponding parameter thresholds output by the decision branch; When the decision parameter is greater than the parameter threshold, it is determined that local obstacle avoidance is required; otherwise, it is determined that local obstacle avoidance is not required.

4. The mobile robot path planning method based on deep reinforcement learning according to claim 1, characterized in that: When the decision branch determines that the local obstacle avoidance condition is not met, the parameter branch does not generate obstacle avoidance parameters.

5. The mobile robot path planning method based on deep reinforcement learning according to claim 1, characterized in that: When the decision branch determines that the local obstacle avoidance condition is not met, the parameter branch does not output the obstacle avoidance parameter.

6. The mobile robot path planning method based on deep reinforcement learning according to claim 3 is characterized in that: Adaptively adjusting the parameter threshold based on the average missed detection rate and the average efficiency loss includes: Obtain the average missed detection rate and average efficiency loss within a set period; use the ratio of the average missed detection rate to the standard ratio as the missed detection coefficient, and use the ratio of the average efficiency loss to the standard loss as the loss coefficient; The basic value is corrected based on the missed detection coefficient and the loss coefficient to obtain the updated parameter threshold.

7. The mobile robot path planning method based on deep reinforcement learning according to claim 3 is characterized in that: Adaptively adjusting the parameter threshold based on obstacle density includes: Extract the base value and the standard density corresponding to the base value; when the obstacle density is the standard density, the local obstacle avoidance capability is optimal when the parameter threshold is set to the base value; Obtain the obstacle density, and use the ratio of the obstacle density to the standard density as the density coefficient; correct the basic value based on the density coefficient to obtain the corrected parameter threshold.

8. The mobile robot path planning method based on deep reinforcement learning according to claim 3 is characterized in that: Adaptively adjusting the parameter threshold based on obstacle density, average missed detection rate, and average efficiency loss includes: Extract the base value and the standard ratio, standard loss, and standard density corresponding to the base value. The standard ratio and standard loss are the average missed detection rate and average efficiency loss corresponding to the base value obtained during the test. When the obstacle density is the standard density, the local obstacle avoidance capability is optimal when the parameter threshold is set to the base value. Obtain the average missed detection rate and average efficiency loss within the set period, as well as the current obstacle density; use the ratio of the average missed detection rate to the standard ratio as the missed detection coefficient, the ratio of the average efficiency loss to the standard loss as the loss coefficient; and the ratio of the obstacle density to the standard density as the density coefficient; The basic value is corrected based on the density coefficient, missed detection coefficient and loss coefficient to obtain the updated parameter threshold.

9. The mobile robot path planning method based on deep reinforcement learning according to claim 2, characterized in that: The acquisition of the priority coefficient includes: Extract the shortest collision time between the mobile robot and the obstacle and the remaining time of the work task; The inverse of the product of the shortest collision time and the remaining time is mapped into a standard range, and the mapping result is used as the priority parameter; wherein the standard range is [0, 1].

Images