Intelligent obstacle avoidance reinforcement learning algorithm for robot in narrow area

Abstract

According to the intelligent obstacle avoidance reinforcement learning algorithm for the robot in the narrow area, more than ten kinds of common narrow terrains are added in a training map, path points are arranged on the two sides of a target terrain, and a reward function with the robot accurately penetrating through the path points as a core is designed; the problems of high complex terrain collision rate and path planning failure caused by model approximation and single training environment in the prior art are solved. Depth camera data is compressed into a 56 * 80-dimensional feature matrix, ultrasonic sensor data is extracted into a 12-dimensional distance vector, multi-modal data fusion training is performed through a three-layer perceptron network, and a 6-dimensional continuous control quantity including three translational speeds and three angular speeds is directly output. The problems of inaccurate size constraint perception and rough action space caused by dependence on a simplified model in a traditional algorithm are solved, refined control over six-degree-of-freedom motion of the robot in a narrow area is achieved, and the obstacle avoidance safety and the path passing rate in a complex obstacle environment are improved.

Description

Technical Field

[0001] The present invention relates to the field of robot intelligent control technology, and in particular to a reinforcement learning algorithm for intelligent obstacle avoidance of robots in narrow areas, which is used to solve the problem of autonomous obstacle avoidance of robots in narrow areas in complex obstacle environments, and provide technical support for safe and efficient navigation of robots in confined spaces. Background Art

[0002] The core difficulty in implementing obstacle avoidance technology for robots operating in confined areas lies in the conflict between physical space constraints and perception and decision-making capabilities. The high ratio of robot size to channel width (e.g., over 80%) renders traditional safety distance control ineffective. Sensors in confined environments are susceptible to interference from specular reflections and visual occlusion (e.g., a LiDAR has a 15° blind spot on vertical walls). Existing technologies generally employ simplified circular / rectangular models (e.g., a virtual circle with a radius enlarged by 1.5 times), but fail to properly assess the collision risk when traversing complex obstacles (e.g., L-shaped corners or narrow corridors, where the radius of curvature is relatively small). Furthermore, low sensor fusion efficiency (e.g., the high latency of asynchronous fusion between LiDAR and camera), insufficient model generalization (simulation-trained models that fail to consider a wide range of scenarios have a low success rate in real-world narrow channel obstacle avoidance), and poor real-time performance further limit the robot's adaptability in dynamic, confined environments.

[0003] Existing technologies often use obstacle avoidance algorithms based on lidar and deep reinforcement learning (DRL). Their core architecture integrates multimodal perception and intelligent decision-making. While these technologies generally perform well in simulation environments, they suffer from the following drawbacks in real narrow passages (such as tunnels with sudden cross-section changes): (1) Insufficient generalization ability: There are few terrain settings and it cannot generalize in complex and narrow scenes; (2) Real-time bottleneck: The inference frequency of the feature extraction network on the embedded platform is lower than 20 Hz, which cannot cope with dynamic obstacles (such as moving obstacles) in real time; (3) Lack of safety redundancy: Relying on a single DRL model for decision-making, it is easy to fall into a local optimal path when the lidar is blocked.

[0004] Existing technologies achieve basic obstacle avoidance capabilities in narrow areas through multimodal perception and reinforcement learning, but breakthroughs are still needed in terms of robustness, real-time performance, and safety in complex dynamic scenes. Therefore, a reinforcement learning algorithm for intelligent obstacle avoidance in narrow areas is urgently needed. Summary of the Invention

[0005] The purpose of the present invention is to overcome the above-mentioned shortcomings and provide a reinforcement learning algorithm for intelligent obstacle avoidance of robots in narrow areas. In order to solve the problem that the simulation training terrain in the existing technology is single and leads to insufficient model generalization, the generalization ability and control accuracy of the reinforcement learning algorithm in complex and narrow environments are improved.

[0006] The object of the present invention is achieved like this: A reinforcement learning algorithm for intelligent obstacle avoidance of robots in narrow areas, including the following: S1, complex and narrow terrain modeling; Build a 3D training environment containing 12 typical narrow scenarios. Within the ISSAC GYM or Genesis engine, these scenarios cover three core categories: linear constraints, geometric constraints, and dynamic constraints. This approach combines programming modeling with point cloud scanning to accurately restore real-world environment parameters. Spatial constraint parameters: channel width is set to 0.5-1.2 meters, curvature radius is 0.3-2 meters, and obstacle height is 1-3 meters; Terrain feature encoding: Generate metadata files containing channel centerline coordinates, safety boundary vectors, and curvature change rates for each terrain type, which serve as prior knowledge input for the reinforcement learning environment; S2, waypoint guided reward function design; Set up hemispherical virtual path points with a diameter of 0.2 meters at both ends of the target terrain and build a multi-stage reward mechanism: R = R pass +R safe +R efficiency , R pass is the path through the reward, R safe is the safe distance bonus, R efficiency是 It is an efficiency reward; S3, heterogeneous sensor data preprocessing; S3.1. Depth camera data processing: A 1280×720 pixel depth camera is used to focus on a narrow area 2 meters in front of the robot and 1 meter to the left and right through ROI extraction technology, compressing it into a 56×80-dimensional depth matrix. A two-dimensional convolutional layer is introduced to perform feature dimensionality reduction, extract geometric features such as edge contours and channel width change rates, and output a 14×20×32 feature tensor; S3.2, Ultrasonic sensor array: Deploy 12 high-precision ultrasonic sensors to collect obstacle distance data in real time and convert it into a 12-dimensional vector in polar coordinates; Perform median filtering and Kalman filtering on the ultrasonic data to eliminate interference from mirror reflection and multipath effects, and output a smoothed distance sequence; S4, three-layer perceptron fusion network architecture; S4.1, input layer: stitching depth image features and ultrasound distance vector, total input dimension 8972; S4.2, hidden layer: S4.2.1. The first layer is 64-dimensional: LeakyReLU activation function is used to reduce the amount of calculation through sparse connections, focusing on extracting key features such as channel width and obstacle azimuth; S4.2.2, the second layer is 128-dimensional: residual connections are introduced to alleviate the gradient vanishing problem of deep networks and learn the mapping relationship between the spatial distribution of obstacles and the robot's posture; S4.2.3, the third layer is 64-dimensional: the output value range is limited to [-1, 1] through the tanh activation function to adapt to the subsequent control quantity mapping; S4.3, Output layer: Map the 64-dimensional features into 6-dimensional control quantities through linear transformation to meet the six-degree-of-freedom control requirements of the robot's translation, rotation, and pitch / roll adjustment in a narrow space. Furthermore, in step S1, the linear constraint scenarios include straight narrow roads and channels with gradually changing widths; the geometric constraint scenarios include 90° right-angle narrow doors and curved roads; and the dynamic constraint scenarios include tunnels with obstacles moving on one side.

[0007] Furthermore, the programming modeling and point cloud scanning in step S1 are combined as follows: a 3D laser scanner is used to perform point cloud scanning on a real narrow scene to obtain environmental point cloud data with an accuracy of ±2 mm; and then, models of 12 typical narrow terrains are constructed through Python programming using the ISSAC GYM software.

[0008] Furthermore, in step S2, the path passes the reward R pass : When the robot's center coordinates pass through the hemisphere of the waypoint, a reward of +100 is given; if the robot deviates from the horizontal projection radius of the waypoint by 0.3 meters, 0.5 points will be deducted for each centimeter.

[0009] Furthermore, the safety distance reward R in step S2 safe : Based on the real-time measurement of obstacle distances in six directions by ultrasonic sensors, the six directions are front / back / left / right / up / down, and a safety margin function is constructed: ; Among them, d min_req is the minimum safety distance corresponding to the robot body size, w i The lateral weight is 1.5, the front-back weight is 1.0, and the up-down weight is 0.5.

[0010] Furthermore, in step S2, the efficiency reward R efficiency : A linear reward of 0-50 points is given based on the ratio of the time to complete the path to the theoretical shortest time, to prevent the model from falling into a conservative obstacle avoidance strategy.

[0011] Furthermore, the depth matrix in step S3.1 retains a spatial resolution of 0.02 meters per pixel; the two-dimensional convolution layer uses a 3×3 convolution kernel with a stride of 2.

[0012] Furthermore, in step S3.2, the 12 high-precision ultrasonic sensors are distributed in a 180° fan-shaped pattern, with 6 sensors facing forward and 3 sensors facing left and right.

[0013] Furthermore, in step S3.2, the depth image features of 14×20×32 are expanded to 8960 dimensions, and the ultrasound distance vector is 12 dimensions.

[0014] Furthermore, the six-dimensional control quantities in step S4.3 are three translational velocities and three angular velocities. The robot translates along the x / y axis, rotates around the z axis, and pitches / rolls around the x / y axis in a narrow space.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a reinforcement learning algorithm for intelligent obstacle avoidance of narrow area robots, which effectively improves the obstacle avoidance safety and passage efficiency of narrow area robots in complex environments through multi-dimensional technological innovation.

[0016] At the training level, the present invention constructs a simulation environment containing a variety of typical narrow terrains, and combines it with a path point-guided reward mechanism to enable the reinforcement learning model to adapt to narrow scenes with different geometric features and spatial constraints, significantly enhancing the algorithm's generalization ability in untrained complex scenes, and solving the problem of strategy failure caused by the single training environment of traditional methods.

[0017] In terms of the perception system, the present invention adopts a multimodal fusion solution of depth cameras and ultrasonic sensors to break through the perception blind spots and accuracy limitations of a single sensor, and realize the precise identification of environmental features such as obstacle distribution and channel width changes in narrow spaces, providing high-resolution real-time data support for obstacle avoidance decisions.

[0018] In terms of control strategy, the present invention innovatively designs six-degree-of-freedom motion control output, which supports the robot to perform refined actions such as posture adjustment, translation and steering in narrow areas. Combined with the dynamic safety boundary generation algorithm, it greatly compresses the required passage space while ensuring safety, significantly improving the robot's ability to pass through extremely narrow passages.

[0019] In terms of algorithm optimization, this invention enhances the model's robustness to extreme conditions such as sensor noise and dynamic interference through adversarial training and experience replay mechanisms, while improving training efficiency, enabling the robot to quickly generate safe and efficient obstacle avoidance strategies in complex dynamic environments.

[0020] The present invention as a whole achieves a leap from conservative obstacle avoidance relying on simplified models to intelligent passage based on precise perception, providing reliable technical support for scenarios with strict requirements on safety and control accuracy, such as nuclear power maintenance and fire reconnaissance, and effectively promoting the practical application capabilities of robots working in narrow environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flow chart of the reinforcement learning algorithm for the intelligent obstacle avoidance of the narrow area robot of the present invention. DETAILED DESCRIPTION

[0022] To better understand the technical solution of the present invention, the following detailed description is provided with reference to the relevant illustrations. It should be understood that the following specific embodiments are not intended to limit the specific implementation of the technical solution of the present invention; they are merely examples of possible implementations of the technical solution of the present invention. It should be noted that references herein to the positional relationships of various components, such as component A being located above component B, are based on the relative positions of the components in the illustrations and are not intended to limit the actual positional relationships of the components.

[0023] See also Figure 1 , Figure 1 A flowchart of the intelligent obstacle avoidance reinforcement learning algorithm for a robot in a narrow area according to Example 1 is drawn. As shown in the figure, the intelligent obstacle avoidance reinforcement learning algorithm for a robot in a narrow area according to Example 1 includes the following contents: S1, complex and narrow terrain modeling; A 3D training environment containing 12 typical narrow scenarios was constructed within the ISSAC GYM or Genesis engine, covering three core scenarios: linear constraints (straight narrow lanes, channels with gradually varying widths), geometric constraints (90° right-angle narrow doors, curved lanes), and dynamic constraints (tunnels with moving obstacles on one side). This approach uses a combination of programming modeling and point cloud scanning to accurately restore real-world environment parameters. S1.1. Spatial Constraint Parameters: Channel width is set to 0.5-1.2 meters (covering 70%-150% of the width of mainstream legged and wheeled robots), curvature radius is 0.3-2 meters (simulating U-shaped and S-shaped curves), and obstacle height is 1-3 meters (matching scenarios such as industrial plants and underground pipe corridors); S1.1 Terrain feature encoding: Generate a metadata file containing the channel centerline coordinates, safety boundary vector, and curvature change rate for each terrain type as the prior knowledge input of the reinforcement learning environment; S2, waypoint guided reward function design; Set up hemispherical virtual path points with a diameter of 0.2 meters at both ends of the target terrain and build a multi-stage reward mechanism: R = R pass +R safe +R efficiency ; S2.1, Path Passing Reward (R pass ): When the robot's center coordinates pass through the hemisphere of the waypoint, the reward is +100; if it deviates from the horizontal projection radius of the waypoint by 0.3 meters, 0.5 points will be deducted for each centimeter; S2.2, Safe Distance Reward (R safe ): Based on the real-time measurement of obstacle distances in six directions by ultrasonic sensors (front / back / left / right / up / down), a safety margin function is constructed: ; Among them, d min_req is the minimum safety distance corresponding to the robot body size (for example, for a robot with a width of 0.8 meters, the minimum lateral safety distance is set to 0.1 meters), w i is the directional weight (lateral weight 1.5, front-back weight 1.0, up-down weight 0.5); S2.3, Efficiency Reward (R efficiency ): A linear reward of 0-50 points is given based on the ratio of the time to complete the path to the theoretical shortest time, to prevent the model from falling into a conservative obstacle avoidance strategy; This reward function uses geometric constraint quantification and multi-objective optimization to force the agent to learn a motion strategy that balances safety and efficiency, solving the problem of path planning failure caused by traditional reward functions that only focus on collision avoidance. S3, heterogeneous sensor data preprocessing; S3.1. Depth camera data processing: A 1280×720 pixel depth camera was used to focus on a narrow area 2 meters in front of the robot and 1 meter to the left and right using ROI (Region of Interest) extraction technology, compressing it into a 56×80-dimensional depth matrix (maintaining a spatial resolution of 0.02 meters per pixel). A two-dimensional convolutional layer (3×3 convolution kernel, stride 2) is introduced to perform feature dimensionality reduction, extract geometric features such as edge contours and channel width change rate, and output a 14×20×32 feature tensor; S3.2, Ultrasonic sensor array: Twelve high-precision ultrasonic sensors (detection range 0.05-5 meters, accuracy ±1%) are deployed in a 180° fan-shaped pattern (6 forward, 3 left and 3 right). These sensors collect obstacle distance data in real time and convert it into a 12-dimensional vector in polar coordinates. Perform median filtering (window size 5) and Kalman filtering on the ultrasonic data to eliminate interference from mirror reflection and multipath effects, and output a smoothed distance sequence; S4, three-layer perceptron fusion network architecture; Using Multi-layer Perceptron (MLP) to achieve data fusion: S4.1, input layer: concatenates depth image features (14×20×32 expanded to 8960 dimensions) and ultrasound distance vectors (12 dimensions), with a total input dimension of 8972 dimensions; S4.2, hidden layer: S4.2.1, first layer (64 dimensions): uses the LeakyReLU activation function to reduce the amount of computation through sparse connections, focusing on extracting key features such as channel width and obstacle azimuth; S4.2.2, second layer (128 dimensions): introduces residual connections to alleviate the gradient vanishing problem of deep networks and learns the mapping relationship between the spatial distribution of obstacles and the robot's posture; S4.2.3, third layer (64 dimensions): uses the tanh activation function to limit the output value range to [-1, 1] to adapt to subsequent control quantity mapping; S4.3, Output layer: Maps the 64-dimensional features into 6-dimensional control variables (3 translational velocities, 3 angular velocities) through linear transformation, meeting the robot's six-degree-of-freedom control requirements for translation (along the x / y axis), rotation (around the z axis), and pitch / roll adjustment (around the x / y axis) in a narrow space.

[0024] Specific implementation case (using the reinforcement learning algorithm of the above embodiment): (1) Modeling of complex and narrow terrain; 1.1. Terrain data collection: Use a 3D laser scanner (such as the FARO Focus3D) to perform point cloud scanning of real narrow scenes (such as nuclear power pipeline tunnels and ancient building corridors) to obtain environmental point cloud data with an accuracy of ±2mm; Using ISSAC GYM software and Python programming, we built models of 12 typical narrow terrains, including: Linear constraint scenario: straight narrow road with a width of 0.6 meters, and gradient channel with a width of 0.8-0.5 meters; Geometrically constrained scenarios: a 0.7-meter narrow door with a 90° right angle, and a curved road with a curvature radius of 0.5 meters; Dynamic constraint scenario: a tunnel with a single-sided obstacle moving back and forth at 0.3 m / s. (2) Implementation of the pathpoint-guided reward function; 2.1. Physical entity mapping: Deploy a hemispherical collision volume with a diameter of 0.2 meters at the center of both ends of the terrain. Its coordinate system is aligned with the robot's base coordinate system. When the robot's center coordinate (obtained through the IMU) enters the hemisphere, a path passing event is triggered. 2.2 Reward Calculation Process: 2.21. Path Passing Reward: Subscribe to the robot's pose data through the ROS topic. When it detects that it has entered the waypoint hemisphere, a +100 reward signal is sent to the reinforcement learning agent. If the robot deviates from the horizontal projection radius of the waypoint by more than 0.3 meters, the PID controller converts each 1 cm of distance into a -0.5 point reward. 2.22. Safety distance reward: The ultrasonic sensor array (12 evenly distributed in front of the robot, on the left and right sides, with an angle of 15°) outputs distance data in real time, according to the formula The calculation is performed, where the weight of the lateral sensors (3 each on the left and right) is set to 1.5, and the weight of the forward sensor is 1.0, and the data is transmitted to the data processing module via the I2C bus; 2.23. Efficiency Reward: Record the actual time the robot takes to travel from the starting point to the end point, compare it with the theoretical shortest time calculated based on the Dijkstra algorithm, calculate it using a linear function, and update it in real time through a Python script. (3) Sensor hardware deployment and data preprocessing; 3.1. Depth Camera Module 3.11. An Intel RealSense D435i camera is installed at the center of the robot's front end with a pitch angle of 15° to acquire 1280×720 pixel depth images. 3.12. ROI extraction using the OpenCV library: The image captures an area centered on the robot's forward direction, 1 meter to the left and 2 meters in front, and compressed into a 56×80-dimensional depth matrix (resolution 0.02 meters / pixel). This is then transferred to the CPU via USB 3.0. 3.13. Deploy 2D convolutional layers (3×3 convolution kernel, stride 2, 2 layers in total) to perform feature dimensionality reduction, output a 14×20×32 feature tensor, and extract geometric features such as channel edges and width change rates. 3.2 Ultrasonic sensor array: 12 HC-SR04 ultrasonic sensors are distributed in a 180° fan shape: 6 forward (10° apart), 3 left and right (15° apart), with a detection range of 0.05-5 meters and an accuracy of ±1%; Data processing flow: Median filtering (window size of 5) to remove impulse noise → Kalman filtering to predict obstacle trajectory → conversion to a 12-dimensional distance vector in polar coordinates, which is then sent to the main controller via the microcontroller at 100 Hz. (4) Implementation of a three-layer perceptron network; 4.1 Hardware Architecture The main controller uses NVIDIA Jetson AGX Orin, which integrates a 6-core ARM CPU and a 2048-core GPU and supports FP16 floating-point operations; 4.2. Network layer specific parameters: 4.21. Input layer: Concatenates the depth image features (14×20×32 expanded to 8960 dimensions) and the ultrasound distance vector (12 dimensions), resulting in a total input of 8972 dimensions. A dropout layer (retention rate 0.8) is used to prevent overfitting. 4.22, Hidden Layer: The first layer (64 dimensions): LeakyReLU activation function (negative slope 0.2), sparse connection (connectivity 60%), focusing on extracting channel width and obstacle azimuth features; The second layer (128 dimensions): Residual connection, batch normalization, learning the mapping relationship between the spatial distribution of obstacles and the robot's posture (fused by IMU + visual odometry); The third layer (64 dimensions): tanh activation function (output range [-1, 1]), adapted to the subsequent kinematic model input; 4.23. Output layer: Linear transformation layer (6-dimensional), outputs 3 translational velocities (vx, vy, vz) and 3 angular velocities (rx, ry, rz), and sends them to the robot controller.

[0025] Working principle: This paper provides a reinforcement learning algorithm for intelligent obstacle avoidance in narrow areas for robots. By adding more than ten common narrow terrain types to the training map, such as S-shaped tunnels, narrow doors with corners, and variable-section passages, and setting waypoints on both sides of the target terrain, the algorithm designs a reward function centered on the robot's precise passage through the waypoints. This algorithm addresses the existing problems of high collision rates and path planning failure in complex terrain caused by model approximation and a single training environment. To address the shortcomings of existing technologies such as low sensor fusion efficiency and insufficient control output dimensionality, the present invention compresses depth camera data into a 56×80-dimensional feature matrix and extracts ultrasonic sensor data into a 12-dimensional distance vector. Multimodal data fusion training is performed through a three-layer perceptron network (64×128×64), and six-dimensional continuous control quantities including three translational velocities (vx, vy, vz) and three angular velocities (rx, ry, rz) are directly output. This solves the problems of inaccurate dimensional constraint perception and rough motion space caused by traditional algorithms relying on simplified models (such as circular expansion radius), realizes refined control of the robot's six-degree-of-freedom motion in narrow areas, and improves obstacle avoidance safety and path pass rate in complex obstacle environments.

[0026] The above are only specific application examples of the present invention and do not constitute any limitation on the scope of protection of the present invention. Any technical solutions formed by equivalent transformation or equivalent replacement shall fall within the scope of protection of the present invention.

Claims

1. A reinforcement learning algorithm for intelligent obstacle avoidance of robots in narrow areas, characterized by: Includes the following: S1, complex and narrow terrain modeling; Build a 3D training environment containing 12 typical narrow scenarios. Within the ISSAC GYM or Genesis engine, these scenarios cover three core categories: linear constraints, geometric constraints, and dynamic constraints. This approach combines programming modeling with point cloud scanning to accurately restore real-world environment parameters. Spatial constraint parameters: channel width is set to 0.5-1.2 meters, curvature radius is 0.3-2 meters, and obstacle height is 1-3 meters; Terrain feature encoding: Generate metadata files containing channel centerline coordinates, safety boundary vectors, and curvature change rates for each terrain type, which serve as prior knowledge input for the reinforcement learning environment; S2, waypoint guided reward function design; Set up hemispherical virtual path points with a diameter of 0.2 meters at both ends of the target terrain and build a multi-stage reward mechanism: R = R pass +R safe +R efficiency , R pass is the path through the reward, R safe is the safe distance bonus, R efficiency是 It is an efficiency reward; S3, heterogeneous sensor data preprocessing; S3.

1. Depth camera data processing: A 1280×720 pixel depth camera is used to focus on a narrow area 2 meters in front of the robot and 1 meter to the left and right through ROI extraction technology, compressing it into a 56×80-dimensional depth matrix. A two-dimensional convolutional layer is introduced to perform feature dimensionality reduction, extract geometric features such as edge contours and channel width change rates, and output a 14×20×32 feature tensor; S3.2, Ultrasonic sensor array: Deploy 12 high-precision ultrasonic sensors to collect obstacle distance data in real time and convert it into a 12-dimensional vector in polar coordinates; Perform median filtering and Kalman filtering on the ultrasonic data to eliminate interference from mirror reflection and multipath effects, and output a smoothed distance sequence; S4, three-layer perceptron fusion network architecture; S4.1, input layer: stitching depth image features and ultrasound distance vector, total input dimension 8972; S4.2, hidden layer: S4.2.

1. The first layer is 64-dimensional: LeakyReLU activation function is used to reduce the amount of calculation through sparse connections, focusing on extracting key features such as channel width and obstacle azimuth; S4.2.2, the second layer is 128-dimensional: residual connections are introduced to alleviate the gradient vanishing problem of deep networks and learn the mapping relationship between the spatial distribution of obstacles and the robot's posture; S4.2.3, the third layer is 64-dimensional: the output value range is limited to [-1, 1] through the tanh activation function to adapt to the subsequent control quantity mapping; S4.3, output layer: Map the 64-dimensional features into 6-dimensional control quantities through linear transformation to meet the six-degree-of-freedom control requirements of the robot's translation, rotation, and pitch / roll adjustment in a narrow space.

2. The intelligent obstacle avoidance reinforcement learning algorithm for a robot in narrow areas according to claim 1 is characterized by: In step S1, linear constraint scenarios include straight narrow roads and channels with gradually changing widths; geometric constraint scenarios include 90° right-angle narrow doors and curved roads; and dynamic constraint scenarios include tunnels with obstacles moving on one side.

3. The intelligent obstacle avoidance reinforcement learning algorithm for a robot in narrow areas according to claim 1 is characterized by: The combination of programming modeling and point cloud scanning in step S1 is as follows: a 3D laser scanner is used to perform point cloud scanning on a real narrow scene to obtain environmental point cloud data with an accuracy of ±2 mm; then, models of 12 typical narrow terrains are constructed using Python programming using the ISSAC GYM software.

4. The narrow area robot intelligent obstacle avoidance reinforcement learning algorithm according to claim 1 is characterized by: In step S2, the path passes the reward R pass : When the robot's center coordinates pass through the hemisphere of the waypoint, a reward of +100 is given; if the robot deviates from the horizontal projection radius of the waypoint by 0.3 meters, 0.5 points will be deducted for each centimeter.

5. The intelligent obstacle avoidance reinforcement learning algorithm for a robot in narrow areas according to claim 1 is characterized by: Safety distance reward R in step S2 safe : Based on the real-time measurement of obstacle distances in six directions by ultrasonic sensors, the six directions are front / back / left / right / up / down, and a safety margin function is constructed: ； Among them, d min_req is the minimum safety distance corresponding to the robot body size, w i The lateral weight is 1.5, the front-back weight is 1.0, and the up-down weight is 0.

5.

6. The intelligent obstacle avoidance reinforcement learning algorithm for a robot in narrow areas according to claim 1 is characterized by: Efficiency reward R in step S2 efficiency : A linear reward of 0-50 points is given based on the ratio of the time to complete the path to the theoretical shortest time, to prevent the model from falling into a conservative obstacle avoidance strategy.

7. The intelligent obstacle avoidance reinforcement learning algorithm for a robot in narrow areas according to claim 1 is characterized by: The depth matrix in step S3.1 retains a spatial resolution of 0.02 meters per pixel; the two-dimensional convolution layer uses a 3×3 convolution kernel with a stride of 2.

8. The intelligent obstacle avoidance reinforcement learning algorithm for a robot in narrow areas according to claim 1 is characterized by: In step S3.2, the 12 high-precision ultrasonic sensors are distributed in a 180° fan-shaped pattern, with 6 sensors facing forward and 3 sensors facing left and right.

9. The intelligent obstacle avoidance reinforcement learning algorithm for a robot in narrow areas according to claim 1 is characterized by: In step S3.2, the depth image features of the splicing are expanded from 14×20×32 to 8960 dimensions, and the ultrasound distance vector is 12 dimensions.

10. The intelligent obstacle avoidance reinforcement learning algorithm for a robot in a narrow area according to claim 1, characterized in that: The six-dimensional control variables in step S4.3 are three translational velocities and three angular velocities. The robot translates along the x / y axis, rotates around the z axis, and pitches / rolls around the x / y axis in a narrow space.

Images