A mobile robot navigation method

Abstract

The present application relates to the field of mobile robot autonomous navigation and artificial intelligence, and particularly relates to a mobile robot navigation method, which comprises the following steps: constructing an SCAG-TD3 deep reinforcement learning network integrating target guidance gating and residual target injection; establishing an automatic course learning system comprising bidirectional adaptive promotion and demotion and forgetting perception review; constructing a social control barrier function based on an anisotropic ellipse social domain, introducing a forward-looking point model to convert it into a linear inequality constraint, and forming a safety filtering layer; and encapsulating and deploying the trained policy network and the safety filtering layer. The present application combines deep reinforcement learning with social perception safety constraints, solves the problems of insufficient perception focus, low training efficiency and no guarantee of social safety in the prior art in a dynamic crowd environment, and realizes efficient, safe and socially compliant autonomous navigation of the robot.

Description

Technical Field

[0001] This invention relates to the field of autonomous navigation and artificial intelligence technology for mobile robots, specifically a navigation method for mobile robots. Background Technology

[0002] In the modern service-oriented society and intelligent construction fields, autonomous mobile robots are widely used in highly dynamic indoor environments such as commercial offices, medical care, and warehousing logistics to perform complex tasks such as material delivery and guidance inspection. To ensure safe passage through densely populated areas, robots need highly flexible real-time obstacle avoidance and path decision-making capabilities. However, traditional robot obstacle avoidance algorithms typically rely on pre-set environmental maps and fixed rule bases. When faced with real-time changing environmental demands such as random pedestrian movement and high-frequency crowd interaction, they often exhibit insufficient robustness and struggle to achieve flexible dynamic obstacle avoidance.

[0003] Deep reinforcement learning, as an algorithmic system with powerful autonomous decision-making and end-to-end learning capabilities, has shown great potential in the field of autonomous robot navigation in recent years. By constructing a deep mapping between perception and control, robots can learn obstacle avoidance strategies from massive amounts of environmental interaction data. However, existing decision-making models based on convolutional neural networks often treat all sensor beams equally, resulting in insufficient focus on key obstacle avoidance features. Furthermore, navigation target information is easily diluted in deep network architectures, inducing decision fluctuations and target loss in specific scenarios such as narrow corridors. In addition, due to the lack of effective adaptive training mechanisms, traditional methods often encounter convergence bottlenecks in highly dynamic scenarios due to the sparse reward signals caused by the surge in environmental complexity.

[0004] To address the lack of safety guarantees in purely data-driven methods, the Control Barrier Function (CBF), a safety constraint method based on Lyapunov stability theory, has been introduced into the field of robot control. CBF mathematically guarantees that the system state always meets preset safety conditions by imposing inequality constraints on the control inputs, thus providing a verifiable safety boundary for the control actions output by reinforcement learning. However, existing CBF methods typically model pedestrians as static circular obstacles, neglecting the anisotropic social space formed by pedestrian movement speed and orientation, and thus failing to achieve truly socially perceptive safety constraints.

[0005] Therefore, developing a robot navigation method that can balance perception focus, training efficiency, and social safety constraints has significant theoretical and engineering application value. Summary of the Invention

[0006] The purpose of this invention is to provide a mobile robot navigation method to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A mobile robot navigation method includes the following steps:

[0009] Step 1: Construct the SCAG-TD3 deep reinforcement learning network, which integrates a target guidance gating module, a residual target injection structure and a gating cross-fusion module;

[0010] Step 2: Construct a hierarchical training scenario library and initialize an automatic course learning system. The automatic course learning system uses a three-component mixed sampling distribution for task scheduling.

[0011] Step 3: Use the automatic course learning system to train the SCAG-TD3 deep reinforcement learning network, and update the training task through a bidirectional adaptive promotion and demotion mechanism and a forgetting-aware hybrid review sampling mechanism.

[0012] Step 4: Establish a social control barrier function constraint model. The social control barrier function constraint model establishes a velocity-adaptive anisotropic elliptical social safety domain for dynamic pedestrians, and introduces a forward-looking model to transform the safety constraints into linear inequalities with respect to the control input.

[0013] Step 5: Embed a safety filter layer composed of the social control barrier function constraint model at the output of the SCAG-TD3 deep reinforcement learning network. By solving a quadratic programming problem with the goal of minimizing action deviation, output a safety correction action.

[0014] Step 6: Encapsulate the trained policy network and the security filtering layer into an inference module and deploy it to the mobile robot control unit to perform navigation.

[0015] As a further aspect of the present invention: step 1 includes:

[0016] A one-dimensional convolutional neural network is used to extract features from LiDAR data to obtain LiDAR feature vectors.

[0017] The target location information and the action at the previous moment are encoded using a fully connected layer to generate target embedding and action embedding;

[0018] In the Actor network of the SCAG-TD3 deep reinforcement learning network, the target embedding is used as the control signal, and the feature vector of the lidar is spatially adaptively weighted and modulated through a gating mechanism.

[0019] In the deep structure of the Actor network, the target embedding is spliced ​​and fused with the hidden layer features to form a residual target injection structure;

[0020] In the Critic network of the SCAG-TD3 deep reinforcement learning network, the action embedding generates a gating signal, which is then filtered by the environmental state features and fused with the action features to form the gating cross-fusion module.

[0021] As a further aspect of the present invention: the three-component hybrid sampling distribution includes: sampling the main task component of the current main task with the main task sampling probability, sampling the same map review component of the low difficulty sub-stage within the same map with the same map low stage review probability, and sampling the cross-map forgetting perception review component of the unlocked old task with the remaining probability according to the review weight; the sum of the main task sampling probability, the same map low stage review probability and the cross-map forgetting perception review probability is 1.

[0022] As a further aspect of the present invention: the review weight is determined based on the forgetting signal and the task difficulty weight; the forgetting signal is the positive difference between the reference success rate anchor point when the task is unlocked and the real-time success rate within the current sliding window; the task difficulty weight is positively correlated with the map sequence number and sub-stage number of the task.

[0023] As a further aspect of the present invention: the bidirectional adaptive upgrade / downgrade mechanism includes:

[0024] Upgrade Decision: If the success rate of the current main task is greater than or equal to the upgrade success rate threshold for K consecutive times, then the difficulty of the current sub-stage will be increased or the next map will be unlocked.

[0025] Degradation Decision: If the success rate of the current main task is less than the downgrade success rate threshold for L consecutive times, the difficulty of the current sub-stage will be reduced, and the previous map will not be reverted.

[0026] As a further aspect of the present invention: the anisotropic elliptical social safety domain is an elliptical constraint region established with the pedestrian's position as the center and the pedestrian's movement direction and vertical direction as the main axes; the major semi-axis of the elliptical constraint region is dynamically adjusted with the pedestrian's speed, while the minor semi-axis is fixed; the social control barrier function constraint model defines a social obstacle function, and when the robot is located outside the elliptical constraint region, it is determined to be in a safe state.

[0027] As a further aspect of the present invention: the forward viewpoint model is as follows: the social control barrier function constraint is applied to the virtual forward viewpoint directly in front of the robot, so that both the linear velocity and the angular velocity appear in the first derivative of the social barrier function, thereby transforming the safety constraint into a linear inequality with respect to the control input.

[0028] As a further aspect of the present invention: the objective function of the quadratic programming problem is to minimize the deviation between the safety correction action and the proposed action output by the SCAG-TD3 deep reinforcement learning network, and the constraint condition is the linear inequality constraint of the social control barrier function of all nearby pedestrians; if the proposed action satisfies all constraints, it is directly output; otherwise, the feasible safety action closest to the proposed action is output.

[0029] The present invention also provides a mobile robot navigation system, comprising:

[0030] The perception module is used to acquire LiDAR data, target point information, and the robot's own status.

[0031] The SCAG-TD3 deep reinforcement learning network constructed using the mobile robot navigation method described above is used to generate proposed actions based on the data from the perception module.

[0032] The social control barrier function security filtering layer constructed by the mobile robot navigation method described above is used to securely project the proposed action and output the final control command.

[0033] An execution module is used to drive the robot to move according to the final control command.

[0034] Compared with the prior art, the beneficial effects of the present invention are:

[0035] 1. Target-guided gating modulation enables robots to adaptively weight LiDAR features based on the navigation target direction, solving the problem of target information being diluted in deep networks.

[0036] 2. The Critic network employs a gated cross-fusion mechanism, which can more accurately evaluate the potential value of specific actions in specific environmental states, and accelerate the training convergence speed in complex dynamic scenarios.

[0037] 3. The automated course learning system, through a two-way adaptive upgrade and downgrade mechanism and forgetting-aware review sampling, enables the model to smoothly transition to complex scenarios with high population density, effectively overcoming the problem of catastrophic forgetting.

[0038] 4. The social control barrier function safety filter layer provides mandatory safety constraints before the action is executed, and mathematically ensures that the robot neither causes physical collisions nor intrudes into the social psychological space of pedestrians. Attached Figure Description

[0039] Figure 1 This is a block diagram of the overall architecture of the navigation system of the present invention.

[0040] Figure 2This is a diagram of the Actor network and Critic network architecture of the SCAG-TD3 deep reinforcement learning network in this invention.

[0041] Figure 3 This is a schematic diagram of the anisotropic social domain and action projection principle of the social control barrier function in this invention.

[0042] Figure 4 This is a schematic diagram of the layout of the training map and the test map of the present invention. The first row is the training map, the second row is the test map (Map-1 to Map-5), the white area is the passable space, and the black area is the obstacle.

[0043] Figure 5 The following is a comparison chart of typical navigation trajectories of each method on the test map, where (a) is the navigation trajectory of the method of the present invention, (b) is the navigation trajectory of the CADRL method, and (c) is the navigation trajectory of the A*+DWA method.

[0044] Figure 6 The chart shows the quantitative performance comparison of each method on the test map. (a) compares the success rate (SR), (b) compares the average time steps (ATS), and (c) compares the social space intrusion rate (SIR). The height of the bar chart represents the mean, and the error bars represent the 95% confidence interval. Detailed Implementation

[0045] The technical solution of this application will be further described in detail below with reference to specific embodiments.

[0046] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0047] System Architecture

[0048] like Figure 1 The diagram shown is an overall architecture block diagram of the navigation system of the present invention. The system consists of three major functional modules: the SCAG-TD3 deep reinforcement learning network structure layer, the social control barrier function security filtering layer, and the automatic course learning layer.

[0049] Its implementation includes the following steps:

[0050] Step 1: Initialize system parameters and network structure.

[0051] Initialize the SCAG-TD3 neural network parameters, including the Actor network, Actor target network, Critic double Q network, and their corresponding target networks; initialize the experience replay pool; initialize the automatic course learning system and set the current main task T. * =(map1,stage1), establish an independent performance buffer queue for all tasks with a capacity of E, and initialize the three-component mixed sampling distribution parameter p. main and p intra , satisfying p main >p intra And p main +p intra <1.

[0052] Where, p main The sampling probability for the main task ensures sufficient training for the current main task; p intr 'a' represents the probability of reviewing at a lower stage on the same map, to prevent performance crashes after upgrades; the remaining probability (1-p) main -p intra This is allocated to the cross-map forgetting-aware review component for targeted review of unlocked old tasks. The sum of the three is 1, together forming the complete mixed sampling distribution. Constraint p main >p intra Ensure that the primary task training always takes precedence, p main +p intra <1 Ensures that the review component of old tasks is not zero. Capacity E is the sliding window size, meaning that each task independently calculates the execution results of the most recent E episodes for success rate calculation.

[0053] Step 2: Select the current training task and configure the simulation environment through the automatic course learning system.

[0054] The Automated Course Learning (ACL) system, at the beginning of each training episode, uses a three-component mixed sampling distribution p(T) to sample data from the two-dimensional task space T = (map). i stage s The current training task T is selected from the given information. The two-dimensional task space contains 5 training maps and 3 sub-stages, forming 15 training task units. The maps are arranged in ascending order of scene structural complexity, including typical indoor constrained geometric elements such as corridors and intersections. The map index... Arranged in ascending order of scene structural complexity; each map contains 3 sub-stages. The difficulty of each sub-stage increases sequentially: Stage 1 corresponds to 0-2 pedestrians at a speed of 0.3 m / s; Stage 2 corresponds to 3-4 pedestrians at a speed of 0.5 m / s; and Stage 3 corresponds to 5-6 pedestrians at a speed of 0.7 m / s. Training progresses monotonically along the course axis without map reversal. The training map is as follows: Figure 4As shown in the first row, the sampling distribution consists of three parts:

[0055] S21. Main task component: with probability p main Fixed sampling of current main task T * To ensure the continuous progress of learning;

[0056] S22, Similar to the lower-level review components of the map: using probability p intra Sample low-difficulty sub-phases within the same map to prevent performance crashes after upgrades;

[0057] S23, Cross-map forgetting perception review component: Residual probability (1-p) main -p intra According to weight w T Assign to the set of unlocked old tasks The review weights for the old task T are calculated as follows (Equation 1):

[0058] (1);

[0059] In Equation 1, w T Normalized review weights for old task T. To minimize review noise and prevent some old tasks from being forgotten due to long-term lack of sampling; α and β are the forgetting signals F, respectively. T and difficulty weight D T The control ratio coefficient is used to adjust the relative contributions of forgetting level and task difficulty to review priority; D T The value is the product of the map number and stage number of the task; the larger the value, the more difficult the task and the higher the review priority. T The forgetting signal is calculated as follows (Equation 2):

[0060] (2);

[0061] In Equation 2 This serves as a reference success rate anchor point recorded when unlocking a task, reflecting the historical best performance at that time. This represents the real-time success rate calculated within the current sliding window; the difference between the two reflects the degree of performance degradation of the model on this task; max(0,·) ensures that the forgetting signal is non-negative, meaning that the review mechanism is triggered only when performance degrades. Based on the selected task T=(map i stage s In the simulation environment, the corresponding map is loaded, and the robot's initial pose, target point position, and stage are randomly generated. s The initial state of a dynamic pedestrian.

[0062] The complete training scheduling process of the above-mentioned automated course learning system is as follows:

[0063] Input: Task space M = {map1, ..., map5}, number of sub-stages S, buffer capacity E, upgrade threshold τ up Degradation threshold τ down Number of consecutive judgments K, L, and sampling parameter p main p intra , α, β, ε.

[0064] Output: The parameters θ of the trained Actor network.

[0065] 1. Initialize the main task T * ←(map1, stage1), each map contains 3 sub-stages. s The difficulty increases sequentially from ∈{1,2,3}.

[0066] 2. Initialize an independent performance buffer for all tasks T. Reference anchor point .

[0067] 3. Sample the training task T according to the three-component distribution p(T), with probability p main Select main task T * With probability p intra Select lower-stage missions on the same map, based on probability. By weight Select the old task.

[0068] 4. Load task T in IR-SIM, run episode, and obtain the result.

[0069] 5. Record the result to ,calculate .

[0070] 6. ifT=T * and |B T* ∣=Ethen.

[0071] 7. if Then, K times consecutively.

[0072] 8. Stage cur ++; If the map meets the requirements, unlock the next map and save the progress. .

[0073] 9. elseif Then, L times consecutively.

[0074] 10. Stage cur --.

[0075] 11. endif.

[0076] 12. endif.

[0077] 13. if the fixed update interval is reached then.

[0078] 14. For all calculate Renormalize sampling weights .

[0079] 15. endif.

[0080] 16. Repeat steps 3-15 until the highest stage of map5 passes or the training step limit is reached.

[0081] 17. returnθ.

[0082] Step 3: Collect multimodal sensor data at the current moment to form a state vector.

[0083] S31. LiDAR Data Acquisition: The robot acquires 360° environmental scanning data at the current moment through LiDAR. It collects 180 distance values ​​at a sampling interval of 2° to form a 180-dimensional LiDAR input vector. The value of each dimension is the distance value of the nearest obstacle in the corresponding direction. The scanning range is truncated to the preset maximum range.

[0084] S32. Target point information calculation: Based on the robot's current position coordinates (x... r ,y r ) and target point coordinates (x g ,y g The relative distance to the target is calculated using the Euclidean distance formula. The target yaw angle θ = atan2(y) is calculated using the arctangent formula. g -y r ,x g -x r ),Will The vectors are concatenated to form a 3D target vector.

[0085] S33, Action History Record: Read the linear velocity v executed in the previous moment. t-1 and angular velocity ω t-1 This forms a 2D action history vector. The 180-dimensional LiDAR data, the 3D target information vector, and the 2D action history vector are then concatenated to form a 185-dimensional global state vector s.

[0086] Step 4: Input the state vector into the SCAG-TD3 Actor network to generate the proposed control action URL.

[0087] like Figure 2As shown, the forward propagation process of the Actor network includes the following sub-steps:

[0088] S41. LiDAR Feature Extraction: The LiDAR data is expanded into a tensor of (Batch, 1, 180), and local perceptual features are extracted sequentially through a three-layer one-dimensional convolutional neural network (1D-CNN) (kernel sizes are 8, 8, and 4, strides are 4, 4, and 2, and the number of channels is 1→4→8→4). After flattening, a 16-dimensional LiDAR feature vector F is output. lidar ;

[0089] S42, Target and Action Embedding: Map the target information vector (3D) to a 16D target embedding F through a fully connected layer. goal The action vector from the previous time step is activated by LeakyReLU; the 2D action vector is then mapped to a 16D action embedding F through a fully connected layer. act Activated by LeakyReLU;

[0090] S43, Target-Guided Gated Modulation (GCG Module): F goal After linear transformation W q Then, a gating weight vector `query` is generated using the tanh activation function; the element-wise product of `Flidar` and `query` is calculated, followed by a linear transformation `W`. g The sigmoid activation function is used to generate a spatially adaptive gate signal; the original lidar features are then weighted and modulated to obtain the target-guided weighted features F. weighted The calculation formula is as follows (Equation 3):

[0091] (3);

[0092] In Equation 3, ⊙ represents element-wise multiplication (Hadamard product), where W q and W g Both are learnable linear transformation matrices, W q F goal Mapped to F lidar The same feature dimensions; query is the gate weight vector, representing the degree of attention the target direction receives from each feature dimension of the LiDAR; gate is the spatial adaptive gate signal, with a value range of (0,1), used to control F. lidar Selective enhancement or suppression of each dimension; F weighted The target-guided weighted lidar features are used as input for subsequent fusion layers.

[0093] S44, First Fusion Layer and Layer Normalization: F weighted F goal F actThe vectors are concatenated into a 48-dimensional vector, mapped to 400 dimensions through a fully connected layer (48→400), and then activated by layer normalization (LayerNorm) and LeakyReLU to obtain a 400-dimensional hidden layer feature x1. Layer normalization effectively alleviates the gradient instability problem under multimodal input.

[0094] S45. Residual Target Injection and Action Output: Embedding hidden layer features x1 and the target into F goal The vectors are concatenated into a 416-dimensional vector, then fused again through a fully connected layer (416→300), activated by LeakyReLU, and passed through an output layer (300→2) with a tanh function to output a normalized proposal action u. rl =(v rl ,ω rl ), where v rl For the normalized linear velocity, ω rl This is the normalized angular velocity. Target information is injected a second time here to ensure that target orientation is not lost in deep decision-making. During the training phase, to encourage exploration, the target information is directed towards u. rl Superimposed truncated Gaussian noise.

[0095] Step 5: Apply a safe projection to the proposed action using the Social Control Barrier Function (Social-CBF) safety filter layer, and output the final safe action u. * .

[0096] like Figure 3 The diagram shown illustrates the anisotropic social domain and action projection principle of social CBF. The workflow of the security filtering layer is as follows:

[0097] S51. Pedestrian social domain modeling: Read the real-time state of all neighboring pedestrians j within the perception range, including their positions (x, y ... j ,y j ), velocity vj and yaw angle θ j .like Figure 3 As shown in the calculation module, pedestrian status As input, the ellipse social domain modeling process outputs the coordinates of the ellipse center point and the lengths of its major and minor semi-axes, 'a'. j and b j The boundary equations and specific modeling process are as follows: For each pedestrian j, their social exclusion zone is modeled as an anisotropic elliptical region. Anisotropy refers to the asymmetry of the pedestrian's social space in different directions: humans are significantly more sensitive to those approaching from directly in front than from the sides and behind. Therefore, the social exclusion zone is larger in the direction of the pedestrian's movement and smaller in the vertical direction and behind. This characteristic is reflected by the difference between the major and minor semi-axes of the ellipse. The social barrier function h is defined. j (x) is as follows (Equation 4):

[0098] (4);

[0099] when At that time, the robot was located outside the social restricted area of ​​pedestrian j and was in a safe state; when At that time, the robot had already intruded into the restricted area. (x) r , y r (x) represents the robot's current position. j , y j ) represents the pedestrian's location, a j and b j These are the semi-axis lengths along the direction of pedestrian movement and the perpendicular direction, respectively. The social barrier function is mathematically represented by an ellipse equation, where the major semi-axis a... j Accompanying speed v j The dynamic increase in size causes the restricted area to extend dynamically along the direction of pedestrian movement. Considering the anisotropic nature of human sensitivity to intrusions from directly in front in social behavior, the restricted area physically exhibits an elliptical distribution, wider in the front and narrower in the back. Figure 3 The anisotropic elliptical region is shown.

[0100] S52. Velocity Adaptive Half-Axis Calculation: Pedestrian Motion Direction Major Half-Axis a j Based on the pedestrian speed v j Dynamic adjustment: a j =a0+kv j Where a0 is the basic length, k is the velocity sensitivity coefficient, and the short half-axis b j Maintaining a fixed position. This design allows the robot to automatically maintain a greater safe distance from pedestrians in their direction of travel;

[0101] S53. Linearization of CBF Constraints at the Forward Viewpoint: Since the robot control space is (v, ω), the linearization of h is directly applied at the robot center. j Taking the first derivative would prevent the angular velocity ω from appearing in the constraints (i.e., a relative order problem). Therefore, a forward-looking model is introduced: the CBF constraint is applied at a distance d directly in front of the robot. la The virtual forward viewpoint at h ensures that both control inputs v and ω appear at h. j The first derivative with respect to time forms a linear inequality constraint with respect to u, resulting in a linear constraint coefficient matrix A for each pedestrian j. cbf，j and constraint vector b cbf，j ;

[0102] Specifically, the coordinates of the foreground point are defined as follows: ,in θ represents the robot's current position. r Let d be the robot's facing angle. la Let h be the forward sight distance. For the obstacle function h... j (x) along the control input Taking the first derivative in the direction, where v is the linear velocity and ω is the angular velocity, we can obtain the linear form of the CBF constraint:

[0103] (5);

[0104] in Let v be the CBF attenuation coefficient. This inequality is linear with respect to both control inputs v and ω, and can be rearranged as follows: The standard form can be directly incorporated into the quadratic programming framework for solution.

[0105] S54, QP safety projection solution: Summarize all pedestrian constraints, construct and solve the quadratic programming problem as shown in equation (5):

[0106] (6);

[0107] QP solver outputs safety correction action u * =(v * ,ω * If the proposed action u rl If u satisfies all CBF constraints, then u * =u rl That is, zero interference at the security layer; otherwise, the distance u is automatically calculated. rl Recent feasible safety actions u * Force the robot to slow down or turn, and u * Send to the robot chassis drive actuator.

[0108] Step 6: The robot performs actions in the environment. * It obtains environmental feedback and stores the experience in the replay pool.

[0109] Robot chassis receiver (v * ,ω * The process is executed, and the environment returns the next time-instance state s', the immediate reward r, and the termination flag done. The reward function comprehensively considers the reward for reaching the target, the collision penalty, the reward for reducing the distance towards the target, and the step penalty. The specific reward function settings are as follows:

[0110] (7);

[0111] The sparse reward item is:

[0112] (8);

[0113] The definition of intensive reward items is as follows:

[0114] (9);

[0115] (10);

[0116] (11);

[0117] (12);

[0118] (13);

[0119] Where ω1=0.3, ω2=0.2, d t and d t−1 Let α be the distance from the target at the current and previous moments, respectively. t h(x) represents the angular deviation from the target at the current moment, h(x) is the elliptical obstacle function of the Social-CBF, and h0 is the threshold for triggering pedestrian approach penalty.

[0120] The physical meaning and design motivation of each award are as follows: R goalDist and R orient Together, they guide the robot to efficiently approach the goal; R pedAvoid Provides continuous pedestrian approach penalty signals; R social As a soft constraint and a Social-CBF hard constraint, a two-layer social safety mechanism is formed, encouraging strategies to proactively avoid pedestrian social spaces, reducing the frequency of CBF intervention and improving navigation smoothness; R timestep This prompts the agent to complete the navigation task as quickly as possible. The sum of the above-mentioned reward items constitutes the immediate reward r=R at each time step, which, together with the state transition information, forms a quintuple of experience. Stored into the experience replay pool. The original RL proposal action u is stored in the replay pool here. rl Instead of the corrected u * This ensures the correctness of the RL training objective; where s is the current state vector, s′ is the state vector at the next moment after the action is executed, and done is the episode termination flag (true when the target is reached, a collision occurs, or a timeout occurs).

[0121] Step 7: Calculate the success rate SRT of the current task T and trigger the ACL upgrade / downgrade judgment.

[0122] After an episode ends, the result (success, collision, or timeout) is recorded in the independent sliding window buffer of task T. When the main task T... * After the buffer accumulates to E times, the current success rate is calculated (Equation 14); for the main task T * At the same time, it triggers the upgrade / downgrade decision:

[0123] (14);

[0124] Where SRT is the sliding success rate of task T; E is the sliding window size, i.e., the success rate of the most recent E episodes; k is the episode index; success k This represents the navigation result of the kth episode. It is true when the robot successfully reaches the target position within the specified time step without a collision, otherwise it is false; 1(·) is an indicator function, which takes the value 1 when the input is true and 0 otherwise.

[0125] S71, Upgrade Judgment: If SR T* ≥ Upgrade threshold τ up If it is true K times consecutively, then stage will be executed. cur ++; where stage cur The stage number is the sub-stage number of the current main task, with a value range of {1, 2, 3}. cur ++ indicates that the difficulty of the current sub-stage is increased by one level, stage cur --Indicates reducing the difficulty of the current sub-stage by one level. If all stages of the current map have been completed, the next map is unlocked, stage 1 is reset, and the current SR is reset. T Recorded as a reference performance anchor for this task. ;

[0126] S72, Downgrade Judgment: If the current success rate SR T* Degradation threshold τ down If it is established L times consecutively, only stage will be executed. cur --, without reverting to the old map, the performance recovery of the old map is handled by the forgetting-aware mixed review sampling mechanism in step 2, and the course axis remains monotonically progressive.

[0127] Step 8: When the number of replay pool experiences exceeds the preheating threshold, start SCAG-TD3 network training and optimize the Actor and Critic parameters.

[0128] Every fixed number of steps, a batch of data is randomly sampled from the experience replay pool, and the following network update is performed:

[0129] S81, Critic Network Update: The target Actor network is used to generate the target action for the next state s' (with truncated Gaussian noise superimposed to smooth the target). The Bellman target is constructed by taking the smaller Q value through the target Critic double Q network, as shown in Equation (15):

[0130] (15);

[0131] in This is a discount factor used to balance immediate rewards with future earnings; (1−done) ensures that future earnings are not calculated in the terminated state.

[0132] Calculate the current predictions Q1(s,a) and Q2(s,a) of the dual-Q network, and minimize the sum of the mean squared error losses: The mean squared error loss function is MSE, which optimizes the Critic parameter by minimizing the error between the double Q network prediction and the Bellman objective. The gated cross-fusion mechanism filters the state features based on the action features to improve the accuracy of Q-value estimation.

[0133] S82, Actor network delayed update: every policy freq Each step executes an Actor update, where policy freq The delayed update interval for the Actor network, i.e., performing one Actor update after every certain number of Critic updates, helps stabilize the training process. The policy gradient is calculated using a single-path Q1 network to maximize the expected Q-value. ,in For the expectation operator, The action output by the current policy network in state s. This is the value estimate of the state-action pair for the Critic network.

[0134] S83, Target Network Soft Update: Perform a soft update on the Actor and Critic target networks, updating the coefficients. : , where θ is the current network parameter, θtarget is the target network parameter, and τ∈(0,1) is the soft update coefficient. A smaller τ value makes the target network slowly follow the current network, ensuring training stability.

[0135] Step 9: Update the old task forgetting signal at fixed intervals and dynamically adjust the mixed review sampling weights.

[0136] At fixed intervals, check all unlocked old tasks. (in Update the current success rate (for all historical missions unlocked before the current main mission). Calculate the forgotten signal FT=max(0, - ).

[0137] If the cumulative number of samples for a certain old task T is less than the minimum sampling threshold E min (E) min To ensure the minimum sample size requirement for statistical reliability of the forgotten signal, the update of the forgotten signal for that task is skipped, as it is considered insufficient. The review weights w for each old task are recalculated according to Equation 1. TThe sampling probabilities are normalized to ensure that review resources are tilted towards the most forgotten and most difficult historical tasks. The strategy for selecting review stages for old maps is to select the highest unlocked stage of the map with a higher probability. (in (The highest sub-stage number currently unlocked on map m) is selected with a relatively low probability. (If it exists), it balances maintaining the strongest capabilities with boundary stability.

[0138] Step 10: Repeat steps 2 through 9 until the training termination condition is met.

[0139] Repeat steps 2 through 9. The success rate of the main mission corresponding to the highest stage of the final map must meet: SR T* ≥τ up

[0140] Training stops when the condition is met K times consecutively, or when the training steps reach a preset limit, and the model parameters of the Actor and Critic networks are saved. The trained Actor network is encapsulated as an independent inference module, and the entire process is verified in a simulation environment. The network can be further deployed on a mobile robot platform with appropriate sensor configurations; during deployment and operation, the Social-CBF safety filter layer runs continuously online, performing safety projections on the output actions of each control cycle in real time, ensuring the robot's safety constraints on pedestrian social spaces throughout the entire operation.

[0141] To verify the effectiveness of the method proposed in this invention, comparative and ablation experiments were conducted on the IR-SIM simulation platform. The scene size was 10m × 10m, and pedestrians were simulated using the ORCA model and set to non-cooperative mode. A total of 5 test maps were used, such as... Figure 4 As shown in the second row, none of the tests were used in training. Each map was run independently for 100 episodes, using the same random seed to ensure reproducibility. All tests were conducted at the highest difficulty level (stage 3). The recorded metrics included: success rate (SR), percentage of episodes with collisions (CR), average time steps (ATS), and social space intrusion rate (SIR).

[0142] Figure 5 (ac) respectively illustrates the method of the present invention, CADRL and A * +DWA demonstrates a typical navigation trajectory comparison in the test scenario. The method of this invention (solid line) presents a smooth target-oriented path, employing tangential detours near pedestrians and remaining outside the pedestrian social boundary throughout, successfully reaching the target. While CADRL can successfully navigate, its detour distance near pedestrians is not precise enough, exhibiting a certain degree of social space intrusion. A *+DWA is prone to local oscillations when it encounters both long walls and dense pedestrians, which can lead to social space intrusion. Furthermore, experiments have shown that at high speeds, due to sampling frequency limitations, DWA may not be able to avoid pedestrians who suddenly turn around, demonstrating the limitations of traditional methods in complex dynamic environments.

[0143] like Figure 6 As shown in (a), the method of the present invention achieved an average success rate of 78.77% across 5 test maps, significantly higher than CADRL (65.0%) and A. * +DWA (58.6%). As map difficulty increases, the success rate of the baseline method decreases more significantly, while the method of this invention remains relatively stable, demonstrating the role of the automatic curriculum learning framework in improving generalization ability.

[0144] like Figure 6 As shown in (b), the average cross-map time steps of the method of the present invention are 190.28 steps, which is significantly faster than CADRL and A. * +DWA decreased by approximately 21% and 34%, indicating that the Target Guiding Gating (GCG) mechanism effectively improved the target following efficiency of the strategy.

[0145] like Figure 6 As shown in (c), the social space intrusion rate (SIR) of the method of the present invention has a cross-map mean of 11.35%, which is about one-third of CADRL (30.2%) and about A * +DWA (38.6%), a quarter of which directly verifies the effectiveness of the Social-CBF safety barrier layer in protecting pedestrian social spaces.

[0146] As shown in Table 1, the ablation experiments used CNN-TD3 as a basic variant because each module in this invention starts with the CNN-TD3 network structure as an improvement, gradually adding ACL and Social-CBF modules, which can more clearly reflect the independent contribution of each component. It is worth noting that the CNN-TD3 baseline (SR: 61.44%) is similar in performance to CADRL (SR: 65.0%) in the main experiment, indicating that the difficulty benchmarks of the two sets of experiments are consistent, and the ablation results corroborate the conclusions of the main experiment.

[0147] ACL: Compared to the basic CNN-TD3, the success rate increased from 61.44% to 69.34% after introducing ACL, the average time steps decreased from 238.67 to 220.14, the collision rate decreased from 24.31% to 18.76%, and the social intrusion rate decreased from 38.52% to 31.47%. This improvement demonstrates that the forgetting-aware training strategy can effectively accelerate the convergence of the strategy in complex and dynamic scenarios, and alleviate catastrophic forgetting by reviewing old tasks, thereby maintaining more stable navigation performance in high-difficulty scenarios and exhibiting stronger generalization performance in various environments.

[0148] Social-CBF: After further introducing the Social-CBF safety layer on top of +ACL (+SC variant), the social intrusion rate dropped significantly from 31.47% to 18.64%, a decrease of approximately 41%, and the collision rate also decreased from 18.76% to 15.23%. This result directly verifies the key role of Social-CBF in social safety constraints: by solving the quadratic programming problem in real time at each control step, the robot's actions are projected into the safety set, ensuring respect for pedestrian social space. Simultaneously, the success rate further increased to 74.42%, indicating that the intervention of the safety layer did not sacrifice navigation efficiency; on the contrary, it indirectly improved the overall success rate by reducing the number of collision-terminating episodes.

[0149] The complete model (Ours): After introducing the SCAG-TD3 improved network structure, the success rate reached 78.77%, the average time step decreased to 190.28 steps, the collision rate decreased to 10.17%, and the social intrusion rate decreased to 11.35%, with all indicators reaching optimal levels. This demonstrates that the three structural innovations—GCG target guidance gating mechanism, residual target injection, and gating cross-fusion Critic—can effectively improve the policy's perception of target direction and the accuracy of Q-value estimation, thereby further improving navigation efficiency while maintaining social safety.

[0150] The results of the four variants show a monotonically progressive improvement trend, verifying that each module in the proposed framework has an independent and complementary contribution.

[0151] Table 1 Ablation Experiment Results

[0152]

[0153] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these should also be considered within the scope of protection of the present invention. These will not affect the effectiveness of the implementation of the present invention or the practicality of the patent.

Claims

1. A navigation method for a mobile robot, characterized in that, Includes the following steps: Step 1: Construct the SCAG-TD3 deep reinforcement learning network, which integrates a target guidance gating module, a residual target injection structure and a gating cross-fusion module; Step 2: Construct a hierarchical training scenario library and initialize an automatic course learning system. The automatic course learning system uses a three-component mixed sampling distribution for task scheduling. Step 3: Use the automatic course learning system to train the SCAG-TD3 deep reinforcement learning network, and update the training task through a bidirectional adaptive promotion and demotion mechanism and a forgetting-aware hybrid review sampling mechanism. Step 4: Establish a social control barrier function constraint model. The social control barrier function constraint model establishes a velocity-adaptive anisotropic elliptical social safety domain for dynamic pedestrians, and introduces a forward-looking model to transform the safety constraints into linear inequalities with respect to the control input. Step 5: Embed a safety filter layer composed of the social control barrier function constraint model at the output of the SCAG-TD3 deep reinforcement learning network. By solving a quadratic programming problem with the goal of minimizing action deviation, output a safety correction action. Step 6: Encapsulate the trained policy network and the security filtering layer into an inference module and deploy it to the mobile robot control unit to perform navigation.

2. The mobile robot navigation method according to claim 1, characterized in that, Step 1 includes: A one-dimensional convolutional neural network is used to extract features from LiDAR data to obtain LiDAR feature vectors. The target location information and the action at the previous moment are encoded using a fully connected layer to generate target embedding and action embedding; In the Actor network of the SCAG-TD3 deep reinforcement learning network, the target embedding is used as the control signal, and the feature vector of the lidar is spatially adaptively weighted and modulated through a gating mechanism. In the deep structure of the Actor network, the target embedding is spliced ​​and fused with the hidden layer features to form a residual target injection structure; In the Critic network of the SCAG-TD3 deep reinforcement learning network, the action embedding generates a gating signal, which is then filtered by the environmental state features and fused with the action features to form the gating cross-fusion module.

3. The mobile robot navigation method according to claim 1, characterized in that, The three-component hybrid sampling distribution includes: sampling the main task component of the current main task with the main task sampling probability, sampling the same map review component of the low difficulty sub-stage within the same map with the same map low stage review probability, and sampling the cross-map forgetting perception review component of the unlocked old task with the remaining probability according to the review weight; the sum of the main task sampling probability, the same map low stage review probability and the cross-map forgetting perception review probability is 1.

4. The mobile robot navigation method according to claim 3, characterized in that, The review weight is determined based on the forgetting signal and the task difficulty weight; the forgetting signal is the positive difference between the reference success rate anchor point when the task is unlocked and the real-time success rate in the current sliding window; the task difficulty weight is positively correlated with the map sequence number and sub-stage number of the task.

5. The mobile robot navigation method according to claim 1, characterized in that, The bidirectional adaptive upgrade / downgrade mechanism includes: Upgrade Decision: If the success rate of the current main task is greater than or equal to the upgrade success rate threshold for K consecutive times, then the difficulty of the current sub-stage will be increased or the next map will be unlocked. Degradation Decision: If the success rate of the current main task is less than the downgrade success rate threshold for L consecutive times, the difficulty of the current sub-stage will be reduced, and the previous map will not be reverted. Here, K and L are both preset positive integers.

6. The mobile robot navigation method according to claim 1, characterized in that, The anisotropic elliptical social safety domain is an elliptical constraint region established with the pedestrian's position as the center and the pedestrian's movement direction and vertical direction as the main axes; the major semi-axis of the elliptical constraint region is dynamically adjusted with the pedestrian's speed, while the minor semi-axis is fixed; the social control barrier function constraint model defines a social obstacle function, and when the robot is located outside the elliptical constraint region, it is determined to be in a safe state.

7. The mobile robot navigation method according to claim 6, characterized in that, The forward viewpoint model is as follows: a social control barrier function constraint is applied to a virtual forward viewpoint directly in front of the robot, so that both the linear velocity and angular velocity appear in the first derivative of the social barrier function, thereby transforming the safety constraint into a linear inequality with respect to the control input.

8. The mobile robot navigation method according to claim 1, characterized in that, The objective function of the quadratic programming problem is to minimize the deviation between the safety correction action and the proposed action output by the SCAG-TD3 deep reinforcement learning network. The constraint is the linear inequality constraint of the social control barrier function of all nearby pedestrians. If the proposed action satisfies all constraints, it is directly output; otherwise, the feasible safety action closest to the proposed action is output.

9. A mobile robot navigation system, characterized in that, include: The perception module is used to acquire LiDAR data, target point information, and the robot's own status. The SCAG-TD3 deep reinforcement learning network constructed according to any one of claims 1 to 8 is used to generate proposed actions based on the data from the perception module; The social control barrier function security filtering layer constructed by the mobile robot navigation method as described in any one of claims 1 to 8 is used to securely project the proposed action and output the final control command; An execution module is used to drive the robot to move according to the final control command.

Images