Tracked robot navigation system based on visual identification

Abstract

The invention relates to the technical field of visual identification, in particular to a tracked robot navigation system based on visual identification, which comprises a visual sensor unit used for collecting environment images in real time; the inertial measurement unit is used for acquiring posture and acceleration data of the robot; the main control processor is used for executing a visual SLAM algorithm, generating an environment map and performing real-time positioning; the track driving mechanism controls the robot to move; environment images are collected in real time through the visual sensor unit, the posture and acceleration data of the robot are obtained in combination with the inertial measurement unit, a visual SLAM algorithm is executed through the main control processor, high-precision environment modeling and real-time positioning are achieved, and finally an optimized path planning instruction is executed through the track driving mechanism. The system can sense the motion state and environmental characteristics of the robot in real time, so that the track driving mechanism dynamically adjusts the driving force, and the problem of slipping or vehicle sinking caused by topographic changes is avoided.

Description

Technical Field

[0001] The present invention relates to the field of visual recognition technology, and in particular to a crawler robot navigation system based on visual recognition. Background Art

[0002] The autonomous navigation technology of tracked mobile robots in complex terrain environments (such as field exploration, disaster relief, industrial inspection, and other scenarios) has continued to develop in recent years. Existing solutions generally use visual SLAM (simultaneous localization and mapping) combined with an inertial measurement unit navigation framework to achieve robot pose estimation and path planning through multi-sensor fusion.

[0003] Due to the lack of real-time perception of the physical properties of the ground (such as differences in stiffness and friction coefficients of mud, sand, gravel, etc.), traditional path planning relies solely on geometric obstacle avoidance and ignores dynamic constraints. This makes tracked robots prone to slipping, getting stuck, or improper torque distribution. For example, in muddy terrain, if the torque drive is output according to the hardened road strategy, it may cause one side of the track to idle and induce posture drift, seriously reducing navigation reliability.

[0004] Therefore, in order to address the problem that existing technologies are unable to adaptively and dynamically adjust the driving force to match the characteristics of complex terrain, the present invention proposes a tracked robot navigation system based on visual recognition, which analyzes the surface texture features through deep learning and links the track driving strategy to improve the robot's passability and safety in unstructured environments. Summary of the Invention

[0005] In order to overcome the problem that the existing technology cannot adaptively and dynamically adjust the driving force to match the complex terrain characteristics, the present invention proposes a tracked robot navigation system based on visual recognition.

[0006] The technical solution of the present invention is: a crawler robot navigation system based on visual recognition, comprising: A visual sensor unit, used to collect environmental images in real time; Inertial measurement unit, used to obtain the robot's posture and acceleration data; A main control processor, connected to the visual sensor unit and the inertial measurement unit, for executing the visual SLAM algorithm, generating an environmental map and real-time positioning; The crawler drive mechanism receives the path planning instructions from the main control processor and controls the movement of the robot.

[0007] Preferably, the visual sensor unit uses a binocular camera or an RGB-D camera, which is installed on a pitch-adjustable bracket on the top of the robot. The pitch angle is adjusted by ±15° through a mechanical structure. The robot's acquisition viewing angle covers a horizontal range of 180° in front of it, and the minimum imaging distance is 0.3m.

[0008] Preferably, the navigation process executed by the main control processor includes: performing ORB feature point extraction and cross-frame matching on continuous frame images collected by the visual sensor unit, constructing a sparse feature map based on key frames, and then synchronously fusing the angular velocity and linear acceleration data output by the inertial measurement unit, solving the robot's six-degree-of-freedom posture through a tightly coupled VIO algorithm based on graph optimization, and using a dynamic window method combined with obstacle distance information to generate a collision-free path in real time.

[0009] Preferably, the system also includes a terrain recognition module, which is mounted on the main control processor and analyzes the RGB image input by the visual sensor unit through a pre-trained ResNet-18 convolutional neural network, identifies the surface texture features and classifies them into four terrain types: mud, sand, gravel and hardened road surface, and outputs terrain ruggedness parameters.

[0010] Preferably, the main control processor adjusts the motor torque distribution strategy of the track drive mechanism according to the recognition result of the terrain recognition module, and increases the single-side track torque by 15%-20% when it is identified as muddy ground; and reduces the travel speed to 50% of the rated value and increases the ground clearance when it is identified as gravel ground.

[0011] Preferably, the system also includes an infrared fill light unit, which includes two groups of 850nm wavelength infrared LED arrays symmetrically distributed on both sides of the visual sensor unit. The control circuit is triggered by the light intensity sensor, and the fill light is activated when the ambient illumination is lower than 50lux. The fill light intensity is adjusted with distance.

[0012] Preferably, the track drive mechanism uses dual motors to independently drive differential steering control, and the main control processor calculates the target speed difference between the left and right tracks in real time based on the path tracking deviation value, and corrects the steering of the robot through PID closed-loop control.

[0013] Preferably, the system also includes a wireless communication module, which uses 4G / 5G and WiFi dual-mode transmission protocols to receive the navigation target point coordinates sent by the remote control console, and simultaneously upload the robot's real-time posture, environmental map and terrain recognition results to the cloud monitoring platform.

[0014] Preferably, the main control processor has a built-in emergency braking protocol. When a moving obstacle is detected within 1m ahead through the fusion of the visual sensor and the inertial measurement unit, a maximum reverse torque instruction is immediately sent to the track drive mechanism, causing the robot to completely stop moving within 0.5s and triggering the sound and light alarm device.

[0015] Preferably, the visual SLAM algorithm uses the ORB-SLAM3 open source framework, supports automatic stitching and switching of multiple maps, and achieves drift-free navigation over a range of more than 1 km² by constructing local sub-maps and establishing topological connection relationships.

[0016] Beneficial effects of the present invention: The visual sensor unit collects environmental images in real time, and the robot's posture and acceleration data are obtained in combination with the inertial measurement unit. The main control processor executes the visual SLAM algorithm to achieve high-precision environmental modeling and real-time positioning. Finally, the optimized path planning instructions are executed by the track drive mechanism. Through the data fusion of the visual sensor and the inertial measurement unit, the system can perceive the robot's motion state and environmental characteristics in real time, so that the track drive mechanism can dynamically adjust the driving force to avoid slipping or getting stuck due to terrain changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 What is shown is a schematic diagram of the workflow of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0019] The present invention provides an embodiment: a crawler robot navigation system based on visual recognition, comprising: A visual sensor unit, used to collect environmental images in real time; Inertial measurement unit, used to obtain the robot's posture and acceleration data; A main control processor, connected to the visual sensor unit and the inertial measurement unit, for executing the visual SLAM algorithm, generating an environmental map and real-time positioning; The crawler drive mechanism receives the path planning instructions from the main control processor and controls the movement of the robot.

[0020] Furthermore, the visual sensor unit collects image data of the robot's surrounding environment in real time, and the inertial measurement unit synchronously obtains the robot's attitude angle, angular velocity and linear acceleration information. The main control processor receives and fuses the two types of sensor data, extracts image feature points through the visual SLAM algorithm, constructs a three-dimensional map of the environment and calculates the robot's real-time posture. Based on this, the optimal navigation path is generated and control instructions are sent to the track drive mechanism to drive the left and right tracks to move differentially to achieve precise steering and movement; the present invention significantly improves the posture estimation accuracy in complex environments through a tightly coupled visual-inertial positioning method, and uses real-time image processing capabilities to enable the system to dynamically adapt to lighting changes and dynamic obstacle scenes. At the same time, the modular design ensures the realization of high-performance navigation functions within the limited carrying space of the tracked robot, solving the problems of large positioning drift and poor environmental adaptability of traditional solutions in unstructured terrain.

[0021] Furthermore, the binocular camera or RGB-D camera installed on the pitch-adjustable bracket on the top of the robot adjusts the pitch angle (within a range of ±15°) through a mechanical structure, collects environmental depth image information in real time with a 180° horizontal field of view, and transmits the data to the main control processor for subsequent processing; the present invention realizes multi-angle environmental perception through the adjustable bracket, ensuring complete detection of obstacles from close to far distances of 0.3m, and the 180° wide viewing angle coverage effectively eliminates the visual blind spot in front of the robot. The depth information provided by the binocular / RGB-D camera significantly improves the obstacle distance measurement accuracy (error <2%), providing high-precision environmental data support for subsequent navigation decisions, and solving the navigation safety hazards caused by the narrow field of view and inaccurate ranging of traditional monocular cameras.

[0022] The navigation process executed by the main control processor includes: extracting ORB feature points and matching them across frames on continuous frame images collected by the visual sensor unit, building a sparse feature map based on key frames, and then synchronously fusing the angular velocity and linear acceleration data output by the inertial measurement unit. The robot's six-degree-of-freedom posture is solved through a tightly coupled VIO algorithm based on graph optimization, and a dynamic window method is used in combination with obstacle distance information to generate a collision-free path in real time.

[0023] Furthermore, the main control processor first performs ORB feature point detection and descriptor matching on the continuous image frames input by the visual sensor, and constructs a sparse feature map based on key frames. At the same time, the inertial measurement unit provides angular velocity and acceleration data in real time. The visual feature point observations and IMU pre-integration results are jointly optimized with the tightly coupled visual inertial odometry (VIO) algorithm in a graph-based optimization framework to solve the robot's six-degree-of-freedom posture. The dynamic window method (DWA) algorithm is used in the path planning stage. By sampling multiple candidate trajectories in the robot's velocity space, combined with obstacle distance information and the robot's kinematic constraints, the cost function of each trajectory is evaluated in real time and the optimal path is selected. Finally, the control command is output with a 100ms period.

[0024] The system also includes a terrain recognition module, which is mounted on the main control processor and uses a pre-trained ResNet-18 convolutional neural network to analyze the RGB image input by the visual sensor unit, identify surface texture features and classify them into four terrain types: mud, sand, gravel, and hardened road surface, and output terrain ruggedness parameters.

[0025] Furthermore, the terrain recognition module receives the RGB image input by the visual sensor unit through the pre-trained ResNet-18 convolutional neural network running on the FPGA accelerator. It first performs image normalization and data enhancement preprocessing, and then extracts features layer by layer through a network architecture consisting of 17 convolutional layers and 1 fully connected layer. The first four stages use 64, 128, 256 and 512 3×3 convolution kernels for feature mapping, respectively. Each stage is followed by batch normalization and ReLU activation function. Finally, global average pooling and softmax classifier are used to divide the surface texture features into four types of terrain: mud, sand, gravel, and hardened road surface, and output the terrain ruggedness parameter on a 0-1 scale to the main control processor.

[0026] The crawler drive mechanism uses dual motors to independently drive differential steering control. The main control processor calculates the target speed difference between the left and right crawlers in real time based on the path tracking deviation value, and corrects the steering of the robot through PID closed-loop control.

[0027] Furthermore, the terrain recognition module transmits the terrain classification results output by the ResNet-18 network to the main control processor in real time. When it is identified as mud, the main control processor controls the track drive mechanism to increase the single-side track torque by 15%-20% to overcome the soil viscosity resistance. When it is identified as gravel, it simultaneously executes the operation strategy of reducing the travel speed to 50% of the rated value and raising the ground clearance, and dynamically adjusts the left and right track speed difference through the PID controller to maintain vehicle stability.

[0028] The system also includes an infrared fill light unit, which includes two groups of 850nm wavelength infrared LED arrays symmetrically distributed on both sides of the visual sensor unit. The control circuit is triggered by the light intensity sensor and activates the fill light when the ambient illumination is lower than 50lux. The fill light intensity is adjusted with distance.

[0029] Furthermore, the infrared fill light unit monitors the ambient illumination in real time through a light intensity sensor. When it detects that the ambient illumination is lower than 50 lux, it automatically triggers two sets of 850nm wavelength infrared LED arrays to start fill light. The fill light intensity is dynamically adjusted according to the target distance detected by the visual sensor. The fill light power in the range of 0.5m-10m increases linearly from 5W to 20W, thereby ensuring that the outline of obstacles can be clearly imaged at different distances.

[0030] The system also includes a wireless communication module that uses 4G / 5G and WiFi dual-mode transmission protocols to receive the coordinates of navigation target points sent by the remote control console, and simultaneously upload the robot's real-time posture, environmental map and terrain recognition results to the cloud monitoring platform.

[0031] Furthermore, the wireless communication module establishes a real-time data link between the robot system and the remote monitoring platform through the 4G / 5G and WiFi dual-mode transmission protocol. The main control processor encapsulates the robot's real-time posture data, environmental map information and terrain recognition results into data packets, which are compressed by the protocol stack and transmitted to the cloud through the wireless channel in a 200ms period. At the same time, it receives the navigation target coordinates and control instructions issued by the monitoring platform.

[0032] The main control processor has a built-in emergency braking protocol. When a moving obstacle is detected within 1 meter ahead through the fusion of the visual sensor and the inertial measurement unit, a maximum reverse torque command is immediately sent to the track drive mechanism, causing the robot to completely stop moving within 0.5 seconds and triggering the sound and light alarm device.

[0033] Furthermore, the main control processor monitors dynamic obstacles within a range of 1m in front in real time through the fusion data of the visual sensor and the inertial measurement unit. When it detects that the moving target is at a speed ≥ 0.5m / s and enters the danger zone, it immediately triggers the emergency braking protocol, sends the maximum reverse torque command to the crawler drive mechanism, and activates the sound and light alarm device at the same time.

[0034] The visual SLAM algorithm uses the ORB-SLAM3 open source framework, supports automatic multi-map stitching and switching, and achieves drift-free navigation over a range of more than 1 km² by constructing local submaps and establishing topological connections.

[0035] Furthermore, the main control processor uses the ORB-SLAM3 framework to achieve large-scale scene navigation. First, a multi-threaded architecture is used to parallelize the image data input by the visual sensor. In the tracking thread, ORB feature points are extracted in real time and bag-of-words model descriptors are calculated. In the mapping thread, a sparse feature map is constructed based on the covisibility graph and essential graph optimization. In the loop detection thread, relocalization is achieved using the DBoW2 bag-of-words location recognition algorithm. When the robot enters a new area, a submap is automatically created and topological connections between submaps are established through similarity transformation. During the positioning process, the active submap is dynamically switched based on the matching of feature points in the current field of view. The multi-map management mechanism of ORB-SLAM3 enables the system to support continuous navigation over a range of more than 1 km². The submap stitching technology based on similarity transformation can reduce the accumulated error control. The improved feature matching strategy can improve the success rate of relocalization, and the positioning robustness is significantly enhanced in complex scenarios such as lighting changes and dynamic object interference.

[0036] See also Figure 1 , further, the workflow of the present invention is described, specifically: The visual sensor unit continuously acquires environmental image information with a 180° horizontal field of view through a binocular camera or RGB-D camera installed on the top of the robot. The camera is equipped with an adjustable pitch bracket, enabling it to dynamically adjust the pitch angle of ±15° according to changes in terrain, thereby ensuring clear imaging of obstacles from a close distance of 0.3m to a long distance. The collected raw image data is transmitted to the main control processor via a high-speed data interface for subsequent processing. At the same time, the system's integrated infrared fill light unit monitors the ambient illumination in real time through a light intensity sensor. When the light level drops below 50 lux, it automatically activates the 850nm wavelength infrared LED array and dynamically adjusts the fill light power to ensure imaging quality at different distances.

[0037] After receiving the visual data, the main control processor immediately starts the ORB-SLAM3 algorithm framework. First, in the tracking thread, it performs ORB feature point detection and descriptor calculation on the input continuous image frames. 1000-1500 feature points are extracted from each frame and a bag-of-words model is constructed. At the same time, the inertial measurement unit synchronously provides angular velocity and linear acceleration data at a frequency of 200Hz. After IMU pre-integration processing, these data are input into the graph optimizer based on the G2O framework together with the visual feature observation values, and the six-degree-of-freedom posture of the robot is solved through the tight coupling (VIO) algorithm.

[0038] During mapping and localization, the system handles large-scale scenarios through a multi-map management mechanism. When the robot enters a new area, it automatically creates a submap and maintains the topological relationships of multiple submaps in memory. Each submap contains approximately 100-200 keyframes and their associated 3D map points. When a loop is detected, the DBoW2 bag-of-words matching algorithm is used to identify visited areas, and a similarity transformation algorithm is used to align the current submap with historical submaps. This design enables the system to support continuous navigation over a range of more than 1 km² while keeping the cumulative error within 0.3%.

[0039] The terrain recognition module loads a pre-trained ResNet-18 convolutional neural network to perform real-time analysis of RGB images input by the visual sensor. The network consists of 17 convolutional layers and 1 fully connected layer. The input image is normalized and then passed through each convolutional layer to extract features. The final output is the four terrain classification results: mud, sand, gravel, and hardened road surface, as well as the corresponding ruggedness parameters (0-1 scale). The entire processing time is controlled within 50ms, and the classification accuracy rate reaches over 90%, providing key environmental perception data for subsequent dynamic control.

[0040] Based on the SLAM positioning results and terrain recognition information, the main control processor starts the dynamic window algorithm (DWA) for path planning. This method samples hundreds of candidate trajectories in the robot's velocity space (linear velocity 0-1.5m / s, angular velocity 0-1rad / s). Each trajectory is calculated through an evaluation function to calculate its comprehensive score for approaching the target, avoiding obstacles, and conforming to terrain characteristics. At the same time, the kinematic constraints of the crawler robot are taken into account. Finally, the optimal trajectory is selected and control instructions are generated. The entire planning cycle is controlled within 100ms to ensure that the robot can respond to dynamic environmental changes in a timely manner.

[0041] Control commands are sent to the track drive mechanism via the CAN bus, and differentiated control is performed based on the terrain recognition results. In muddy conditions, the torque of one side of the track is increased by 15%-20% to overcome soil resistance. In gravel terrain, the travel speed is reduced to 50% of the rated value and the ground clearance is increased. Under normal circumstances, the speed difference between the left and right track motors is precisely controlled based on the PID algorithm to achieve trajectory tracking with an error of less than 5cm. When a moving obstacle is detected within 1m in front, the emergency braking protocol is immediately triggered, and the maximum reverse torque is output within 0.5s to bring the robot to a complete stop.

[0042] The wireless communication module maintains connection with the remote monitoring platform through 4G / 5G and WiFi dual-mode links, uploading the robot's position, environmental map, and terrain data in a 200ms cycle. After compression, the average bandwidth occupied by the data does not exceed 500kbps. At the same time, it receives navigation target points or control instructions issued by the monitoring platform.

[0043] Furthermore, the present invention provides an embodiment: In field geological exploration scenarios, a tracked robot equipped with this system uses a binocular camera to collect complex terrain images in real time. The ORB-SLAM3 algorithm constructs a three-dimensional map containing rock and gully features. When the terrain recognition module detects soft sand, the main control processor automatically increases the track torque by 18% and reduces the travel speed. Combined with the obstacle avoidance path planned by the dynamic window method, it successfully crosses a sandy slope with a slope of 30°, maintaining positioning accuracy within 0.3%, and reducing the number of slips by 40% compared to traditional solutions.

[0044] Furthermore, the present invention provides an embodiment: When used for chemical plant inspections, the system triggers the infrared fill light unit in low-light pipeline areas. The 850nm infrared LED illuminates the pipeline flanges and valves within a range of 10m with a power of 15W. The visual SLAM maintains a 95% feature point matching success rate even under the interference of highly corrosive steam. When a sudden leakage obstacle is detected (moving speed 0.8m / s), the emergency braking protocol completes braking within 0.4 seconds, and the sound and light alarm simultaneously activates a 105dB buzzer warning to avoid collision accidents.

[0045] Through the above steps, by fusing data from the visual sensor and the inertial measurement unit, the system can perceive the robot's motion state and environmental characteristics in real time, allowing the track drive mechanism to dynamically adjust the driving force to avoid slipping or getting stuck due to terrain changes, thereby solving the problem that existing technologies cannot adaptively and dynamically adjust the driving force to match complex terrain characteristics.

Claims

1. A crawler robot navigation system based on visual recognition, characterized in that: Includes: A visual sensor unit, used to collect environmental images in real time; Inertial measurement unit, used to obtain the robot's posture and acceleration data; A main control processor, connected to the visual sensor unit and the inertial measurement unit, for executing the visual SLAM algorithm, generating an environmental map and real-time positioning; The crawler drive mechanism receives the path planning instructions from the main control processor and controls the movement of the robot.

2. The crawler robot navigation system based on visual recognition according to claim 1, characterized in that: The visual sensor unit uses a binocular camera or an RGB-D camera, which is installed on a pitch-adjustable bracket on the top of the robot. The pitch angle is adjusted by ±15° through the mechanical structure. The robot's acquisition viewing angle covers a horizontal range of 180° in front of it, and the minimum imaging distance is 0.3m.

3. The crawler robot navigation system based on visual recognition according to claim 1, characterized in that: The navigation process executed by the main control processor includes: extracting ORB feature points and matching them across frames on continuous frame images collected by the visual sensor unit, building a sparse feature map based on key frames, and then synchronously fusing the angular velocity and linear acceleration data output by the inertial measurement unit. The robot's six-degree-of-freedom posture is solved through a tightly coupled VIO algorithm based on graph optimization, and a dynamic window method is used in combination with obstacle distance information to generate a collision-free path in real time.

4. The crawler robot navigation system based on visual recognition according to claim 1, characterized in that: The system also includes a terrain recognition module, which is mounted on the main control processor and uses a pre-trained ResNet-18 convolutional neural network to analyze the RGB image input by the visual sensor unit, identify surface texture features and classify them into four terrain types: mud, sand, gravel, and hardened road surface, and output terrain ruggedness parameters.

5. The crawler robot navigation system based on visual recognition according to claim 4, characterized in that: The main control processor adjusts the motor torque distribution strategy of the track drive mechanism according to the recognition results of the terrain recognition module. When it is identified as mud, the single-side track torque is increased by 15%-20%. When it is identified as gravel, the travel speed is reduced to 50% of the rated value and the ground clearance is increased.

6. The crawler robot navigation system based on visual recognition according to claim 1, characterized in that: The system also includes an infrared fill light unit, which includes two groups of 850nm wavelength infrared LED arrays symmetrically distributed on both sides of the visual sensor unit. The control circuit is triggered by the light intensity sensor and activates the fill light when the ambient illumination is lower than 50lux. The fill light intensity is adjusted with distance.

7. The crawler robot navigation system based on visual recognition according to claim 1, characterized in that: The crawler drive mechanism uses dual motors to independently drive differential steering control. The main control processor calculates the target speed difference between the left and right crawlers in real time based on the path tracking deviation value, and corrects the steering of the robot through PID closed-loop control.

8. The crawler robot navigation system based on visual recognition according to claim 1, characterized in that: The system also includes a wireless communication module that uses 4G / 5G and WiFi dual-mode transmission protocols to receive the coordinates of navigation target points sent by the remote control console, and simultaneously upload the robot's real-time posture, environmental map and terrain recognition results to the cloud monitoring platform.

9. The crawler robot navigation system based on visual recognition according to claim 1, characterized in that: The main control processor has a built-in emergency braking protocol. When a moving obstacle is detected within 1 meter ahead through the fusion of the visual sensor and the inertial measurement unit, a maximum reverse torque command is immediately sent to the track drive mechanism, causing the robot to completely stop moving within 0.5 seconds and triggering the sound and light alarm device.

10. The crawler robot navigation system based on visual recognition according to claim 1, characterized in that: The visual SLAM algorithm uses the ORB-SLAM3 open source framework, supports automatic multi-map stitching and switching, and achieves drift-free navigation over a range of more than 1 km² by constructing local submaps and establishing topological connections.

Images