Hilly and mountainous area tracked vehicle intelligent chassis system and algorithm based on environment perception and road surface recognition

By designing an intelligent chassis system on tracked vehicles in hilly and mountainous areas, and using multimodal perception and fusion algorithms for environmental identification and decision-making control, the problem of insufficient driving efficiency and safety of traditional tracked vehicles under complex terrain and diversified road surface conditions is solved. , achieving more efficient and reliable operation of tracked vehicles.

CN119414830APending Publication Date: 2025-02-11KUNMING UNIV OF SCI & TECH
View PDF 0 Cites 4 Cited by

Patent Information

Application Number
CN202411311007.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-19
Publication Date
2025-02-11

AI Technical Summary

Technical Problem

Under the complex terrain and diversified road surface conditions in hilly and mountainous areas, traditional tracked vehicles have shortcomings in driving and operating efficiency and safety, which makes it difficult for agricultural mechanization and automation to develop.

Method used

An intelligent chassis system for crawler vehicles in hilly and mountainous areas based on environmental perception and road recognition was designed, using industrial CMOS vision camera, four-wire mechanical lidar and 6-axis IMU inertial sensor for environmental perception, combined with FPN+ResNet50 algorithm and IPM image inverse Perspective transformation, 3D sparse convolution graph neural network and Pi-Sigma fuzzy neural network learning algorithm perform multimodal fusion and decision-making control to realize real-time adjustment of track tension and forward angle.

Benefits of technology

It improves the driving stability and passability of tracked vehicles under complex working conditions, reduces track wear, enhances the reliability and application range of vehicles, and promotes the development of intelligent tracked vehicle technology.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119414830A_ABST
    Figure CN119414830A_ABST
Patent Text Reader

Abstract

The invention provides a hilly and mountainous area tracked vehicle intelligent chassis system and algorithm based on environmental perception and road surface recognition, and belongs to the technical field of agricultural mechanical equipment. The system comprises a mechanical structure part and a control system part. According to the algorithm, multi-modal sensing and fusion decision making of the mobile terminal are achieved, the FPN + ResNet50 algorithm, the IPM image inverse perspective algorithm, the 3D sparse convolutional graph neural network, the TSDF truncation signature distance algorithm, the MDU algorithm, the gating modal sensing convolutional network and the Pi-Sigma fuzzy neural network learning algorithm are innovatively combined, the recognition precision and the decision making reliability are guaranteed, and meanwhile, the fusion decision making efficiency of the mobile terminal is improved. And the calculation amount and the calculation cost are reduced, and the recognition efficiency is improved, so that more accurate, timely and reliable recognition perception and feedback control are realized. The algorithm comprises the steps of image acquisition, image feature extraction, feature fusion, feature comparison and the like, and the method is not only suitable for field operation of tracked vehicles in complex hilly and mountainous areas, but also suitable for transfer driving of the tracked vehicles in a working environment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the fields of electronic information technology and agricultural machinery and equipment technology, and in particular relates to an intelligent chassis system for tracked vehicles in hilly and mountainous areas based on environmental perception and road surface recognition. Technical Background

[0002] With the rapid development of automation and intelligence in the field of agricultural machinery, traditional plain agricultural areas have achieved a high level of agricultural automation. However, the development of agricultural automation in the hilly and mountainous areas of the southwest has been frequently hindered due to its complex and dangerous terrain and ground conditions. Traditional tracked vehicles mainly rely on manual operation and driving, which greatly affects the operating efficiency, not only limiting its application scope in complex environments, but also increasing the safety risks of operators.

[0003] There are two main factors that make it difficult for agricultural machinery automation and intelligence to develop smoothly in hilly and mountainous areas. The first is the mechanical part of the vehicle itself. The road environment in hilly and mountainous areas is very diverse, including but not limited to mud, gravel, grass, concrete, etc., which puts higher requirements on the driving safety and stability of the vehicle. Traditional tracked vehicles are mostly bulky, and their chassis structure is fixed at the factory and cannot adapt to the ever-changing road conditions.

[0004] The second is the automated perception and control part. The mountainous environment is complex and changeable, and the characteristics of different types of road surfaces vary greatly, and they will change over time due to weather, human factors, and other reasons. How to effectively distinguish and identify different types of road surfaces and how to update the road surface model in a timely manner to reflect the latest status is a major challenge. At the same time, under severe weather conditions such as rain, snow, and fog that often occur in mountainous areas, the effective working range and accuracy of the sensor will be greatly reduced. The cost of high-performance sensors is relatively high, especially in the case of multi-sensor fusion, the cost of the entire system will increase significantly. How to design an efficient and reliable decision-making algorithm to cope with various complex working conditions is a difficult point.

[0005] How to improve the efficiency and safety of driving and working on unstructured road conditions, as well as the environmental perception ability of vehicles under complex working conditions, has become a key issue that needs to be solved urgently. By solving these key pain points, the development of intelligent tracked vehicle technology can be further promoted, making its application in complex terrains such as hilly and mountainous areas more extensive and efficient. Summary of the invention

[0006] The first purpose of the present invention is to provide an intelligent chassis system for tracked vehicles in hilly and mountainous areas based on environmental perception and road surface recognition. The second purpose of the present invention is to provide an algorithm based on environmental perception and road surface recognition to overcome the existing technical solutions in complex working conditions in hilly and mountainous areas, such as inaccurate environmental perception, poor vehicle passability, and large track wear, which make it difficult for agricultural mechanization and automation to continue to develop in hilly and mountainous areas.

[0007] The first object of the present invention is achieved by:

[0008] Including mechanical structure part and control system part;

[0009] The mechanical structure part includes an engine 1, a gearbox 2, a transmission shaft 3, a crawler track 4, a driving wheel 5, a guide wheel 6, a frame 12, a hydraulic oil pump 13, and a crawler track tension adjustment device and a crawler track inclination adjustment device. The crawler track tension adjustment device includes a pair of tensioning wheels 7, a pair of tensioning wheel swing rods 15, a tensioning wheel connecting rod 16 and a No. 1 servo hydraulic cylinder 14. The height of the tensioning wheel 7 is adjusted by the No. 1 servo hydraulic cylinder 14 to adjust the crawler track tension. The crawler track inclination adjustment device includes a pair of front auxiliary wheels 8, a pair of front auxiliary wheel swing rods 11, a front auxiliary wheel connecting rod 10 and a No. 2 servo hydraulic cylinder 9. The front auxiliary wheel 8 is lifted by the No. 2 servo hydraulic cylinder 9 to adjust the crawler track inclination;

[0010] The industrial CMOS vision camera 20 is two high-definition camera lens modules installed in front of the tracked vehicle chassis, the four-line mechanical laser radar 21 is installed on the top of the tracked vehicle, and the 6-axis IMU inertial sensor 22 is installed inside the chassis;

[0011] The No. 1 servo hydraulic cylinder 14 is installed in the middle of the frame 12 through a bearing base, and its piston rod is connected to the tension wheel connecting rod 16. A pair of tension wheels 7 at both ends of the connecting rod are respectively installed on the upper inner side of the middle of the left and right crawlers 4. The tension wheel connecting rod 16 is connected to a pair of tension wheel swing rods 15 located on both sides of the frame 12, forming a set of slider swing rod mechanism. When the No. 1 servo hydraulic cylinder 14 drives the piston rod to extend and retract, the tension wheels 7 on both sides complete the rising and falling actions synchronously, thereby ensuring that the tightness of the crawlers on both sides is the same. The second servo hydraulic cylinder 9 is installed at the front of the frame 12, and its piston rod is connected to the front auxiliary wheel connecting rod 10. A pair of front auxiliary wheels 8 at both ends of the connecting rod are respectively installed on the front upper inner side of the left and right crawlers 4. The front auxiliary wheel connecting rod 10 is connected to a pair of front auxiliary wheel swing rods 11 respectively located on both sides of the frame 12, forming a set of slider swing rod mechanism. When the second servo hydraulic cylinder 9 drives the piston rod to extend and retract, the front auxiliary wheels 8 on both sides synchronously complete the rising and falling actions, thereby ensuring that the front inclination angles of the crawlers on both sides are consistent;

[0012] The control system consists of a three-layer structure. The first layer is the perception layer, which includes two industrial CMOS vision cameras 20, a four-line mechanical laser radar 21, and a 6-axis IMU inertial sensor 22 to identify the terrain type and landform features that the tracked vehicle is traveling on and about to reach; the second layer is the control layer, which is composed of the edge computer Jetson Xavier NX-T506S main control core 17 as the main control core of the tracked vehicle, and the Robot Operating System 18 as the main control system, responsible for image processing; the third layer is the drive layer, including the servo driver 19 used to receive host signals and send drive signals to the corresponding servo hydraulic cylinders.

[0013] The second object of the present invention is achieved by comprising the following steps:

[0014] Step 1. Use the Jetson Xavier NX-T506S main control core 17 as the main control core of the tracked vehicle, the Robot Operating System as the main control system, the RK3588J as the main control board, and use two industrial CMOS vision cameras 20 to form a binocular vision system, a four-line mechanical laser radar 21 and a six-axis IMU inertial sensor as an environmental perception unit;

[0015] Step 2: When the tracked vehicle is moving, two industrial CMOS visual cameras 20 are used to collect visual images of the road in the forward direction, a laser radar 21 is used to scan the road in the forward direction to establish point cloud information, and a 6-axis IMU inertial sensor 22 is used to record the vehicle acceleration, inclination, impact vibration, rotation and other information in real time, and the obtained data is transmitted back to the Jetson Xavier NX-T506S main control core 17;

[0016] Step 3: For the visual images captured by the industrial camera 20, the ResNet50 algorithm with FPN as the image backbone is used to extract multi-scale image features. The IPM image inverse perspective transformation is used to map the image from the image coordinate system to the world coordinate system to eliminate the interference and error of the perspective effect on the image detection and recognition tasks, and finally obtain the position coordinates of the extracted features. The formula is as follows, where 2α represents the camera image field of view, m and n represent the height and width of the image, h represents the height offset of the camera relative to the object, and u i 、v i Represents the pixel index, x p ,y p Represents image u i 、v i The corresponding estimated distance in the vehicle coordinate system:

[0017]

[0018] For the point cloud information scanned by the laser radar 21, a set of 3D sparse convolutional graph neural networks is used to convert the point cloud into voxels and extract features in the voxel space. The 3D sparse convolutional graph neural network formula is expressed as follows, where x is the feature representation of each node i, y is the label corresponding to the node, A is the adjacency matrix representing the graph, W is the weight matrix, and H is the defined function:

[0019]

[0020] Each convolutional block outputs voxel features of different scales, representing different levels of abstraction of the point cloud, and the multi-scale features are further interacted with their 2D counterparts through the proposed GMA convolutional block.

[0021] The TSDF truncated signature distance algorithm is used to calculate the TSDF value and RGB grayscale value of each voxel. The sdf value of voxel x is calculated as follows:

[0022] sdf(x)=d p -d x =D(I x )-cam z (x) (8)

[0023] The tsdf value of voxel x can be calculated based on its sdf value. The formula is as follows:

[0024]

[0025] Traverse all voxels and calculate the TSDF weight of the current frame. Fuse the current frame with the previous fusion result. Each voxel is updated based on the previous voxel. After weighted average, the results of all voxels are obtained and spliced ​​into a 3D model. The TSDF value calculation and update formula is as follows:

[0026]

[0027] W(p)=W(p)+w(p) (12)

[0028] Step 4: Multimodally fuse the geometric features and semantic features obtained above into the same scene, estimate the depth of the pixel, and then use its semantics of the 3D world to fuse with the real 3D point cloud, regard each image feature pixel as a seed, and generate virtual points in the BEV space. At the same time, the MDU algorithm is used to improve the geometric accuracy and semantic richness of the generated virtual points. The formula is as follows:

[0029]

[0030] At the same time, under the guidance of the reference lidar point cloud voxels, a gated modality-aware convolutional network is used to control and update its camera corresponding target, adaptively select more valuable information parts from the camera voxel features, and then group them for sparse convolution to aggregate them to achieve full multimodal interaction. This gated formula is expressed as:

[0031]

[0032] After executing MDU and gated modality-aware convolutional networks at different scales in the voxel space, corresponding multimodal voxel features are generated at each scale, and then these multi-granular information are further aggregated using progressive cascade connections across different scales. The formula is as follows:

[0033]

[0034] Through the above series of feature extraction and fusion, the multimodal fusion interaction of camera images and lidar point clouds is realized, and the final multimodal voxel features are sent to the recognition prediction module of the main control core as the basis for subsequent road type recognition and judgment;

[0035] Step 5. Compare the multimodal voxel features mentioned above with the pre-trained common road model data set in hilly and mountainous areas, and refer to the real-time vehicle dynamic information in the same frame on the time axis. Use the Pi-Sigma fuzzy neural network learning algorithm for decision control, which is more adaptable to the dynamic characteristics of uncertain road systems and has stronger robustness and fault tolerance. The multimodal voxel features are used as input signals from the input layer to the fuzzification layer. Among them, the input layer is 3 neurons, and each input is fuzzified using 5 membership functions. Assume i = 1, 2, 3, j = 1, 2, 3, 4, 5, then the formula is as follows:

[0036]

[0037] The fuzzified data is processed in the fuzzy reasoning layer, and the 125 fuzzy rules formed by formula (9) are reasoned and output. The reasoning formula of each rule premise is as follows:

[0038]

[0039] The 125 fuzzy rule reasoning outputs formed by formula (9) are as follows: Finally, if the output layer is defuzzified, the final output is obtained. The formula is as follows:

[0040]

[0041] If the output layer gets the expected output, the learning algorithm ends. Otherwise, the gradient descent method is used to adjust the weight values ​​between the layers for back propagation. Finally, the weight learning algorithm of the output layer is obtained, where η is the learning rate and α is the momentum factor. The formula is as follows:

[0042]

[0043] For each iteration of multiple samples, each sample is trained and the weights are updated. After all samples are trained, the next iteration is performed until the requirements are met. The final decision is the road type and distance, as well as the width of the gully or the height of the ridge ahead. This result is sent to the control module in the main control core as a basis for subsequent mode selection;

[0044] Step 6: After the control module obtains the road type determined by the decision, it selects the track tension mode according to the preset table and sends the corresponding command signal to the servo driver 19. After receiving the command signal, the servo driver 19 sends the corresponding control signal to the No. 1 servo hydraulic cylinder 14. The preset tension adjustment reference table is as follows: "+" represents that the hydraulic cylinder is lifted to drive the tension wheel to rise, that is, the tension is tightened; "+" represents that the hydraulic cylinder is retracted to drive the tension wheel to fall, that is, the tension is loosened:

[0045] Preset tension adjustment reference table

[0046]

[0047] If there is a ridge / gully ahead, the calculated ridge height or gully width will be compared with the vehicle parameters. If it cannot pass, an alarm will be given and the vehicle will be taken over by manual remote control. If it can pass, an extension command will be sent to the servo driver 19. At the same time, in order to maintain the height of the tensioning wheel 7 required for the same tension, the corresponding command signal is sent to the servo driver 19, and the servo drivers 19 of each system receive the corresponding control signals and send them to the first servo hydraulic cylinder 14 and the second servo hydraulic cylinder 9. After passing the ridge, a reset signal is sent.

[0048] Step 7, in the tension adjustment mode, the No. 1 servo hydraulic cylinder 14 performs the jacking, lowering, and locking actions of the tension wheel connecting rod 16 according to the received servo signal, driving the tension wheels 7 at both ends to rise and fall, thereby adjusting the track tension. In the caster angle adjustment mode, the No. 2 servo hydraulic cylinder 9 performs the forward pushing, backward pulling, and locking actions of the front auxiliary wheel connecting rod 10 according to the received servo signal, driving the front auxiliary wheels 8 at both ends to rise and fall, thereby adjusting the track caster angle. At the same time, the No. 1 servo hydraulic cylinder 14 synchronously lifts, drops, and locks according to the signal to ensure that the tension of the track 4 remains unchanged while the position of the front auxiliary wheel 8 changes.

[0049] Step 8, the vehicle continues to travel, and the above steps 2 to 7 are repeated until the tracked vehicle reaches the destination. During this period, if an impassable road appears, or if an identification error causes abnormal adjustment of the height of the tensioning wheel 7 or the position of the front auxiliary wheel 8, the vehicle will be parked and an alarm will be issued and handled by manual remote control to improve the robustness of the system. As a preferred embodiment of the present invention, in step 1, the two industrial CMOS vision cameras 20 are cameras installed at a distance of l in front of the tracked vehicle chassis, the four-line mechanical laser radar 21 is installed on the top of the tracked vehicle, and the 6-axis IMU inertial sensor 22 is installed inside the chassis. This layout can maximize the use of each collector, collect the most valuable information while protecting the collector from damage caused by environmental interference.

[0050] As a preferred solution of the present invention, in step 3, the FPN+ResNet50 algorithm and IPM image inverse perspective transform are used to perform multi-scale image feature extraction processing on the two-dimensional visual image, and the 3D sparse convolutional graph neural network and TSDF truncated signature distance algorithm are used to voxelize the three-dimensional point cloud.

[0051] As a preferred solution of the present invention, in step 4, a progressive cascade connection is adopted, assisted by MDU and gated modality-aware convolutional network to obtain multimodal voxel features, thereby realizing multimodal fusion interaction of camera images and lidar point clouds.

[0052] As a preferred solution of the present invention, in step 5, a Pi-Sigma fuzzy neural network learning algorithm is used to make a final decision on all the collected data to obtain the road type and landform feature information ahead.

[0053] As a preferred solution of the present invention, in step 6, the main control unit can select the most appropriate track tension mode according to the road information obtained by identification, thereby improving the grip and power utilization while reducing the wear and even derailment of the track 4. The main control unit can also determine whether the vehicle condition allows passing the obstacle ahead, and if not, stop moving forward, and issue an alarm to prompt manual processing and control.

[0054] The embodiment of the present disclosure includes a multi-modal fusion environment perception system based on a visual camera 20, a laser radar 21 and an IMU sensor, including an independent perception module and a fusion perception module. The perception system includes:

[0055] The independent perception module includes three parts: an industrial CMOS visual camera 20, a four-line mechanical laser radar 21, and a six-axis IMU inertial sensor, as well as some main controls, which are responsible for capturing and preprocessing the vehicle's surrounding environment information;

[0056] The fusion perception module is used to perform multimodal fusion interaction based on the information obtained by the independent perception modules. It uses a series of algorithm networks to obtain the final voxel feature model for reference in subsequent decision-making.

[0057] The disclosed embodiment includes an intelligent chassis design for a tracked vehicle, including control logic and structural design. The chassis design includes:

[0058] Control logic, based on structural analysis and historical advanced experience, summarizes a corresponding table between road type and applicable track tension, and based on calculation simulation fitting, obtains a reliable conversion formula between the extension and contraction amount of the servo hydraulic rod and the height of the tension wheel, providing a logical basis for the control of the tension wheel 7;

[0059] The structural design adds a pair of tension wheels 7 and a pair of front auxiliary wheels 8 on the basis of the conventional tracked vehicle chassis, and each is equipped with a slider rocker mechanism composed of a connecting rod, a rocker arm, and a servo hydraulic cylinder for real-time regulation, thereby regulating the tension and the forward inclination angle of the track 4. At the same time, the servo hydraulic cylinder used has the advantages of high precision, high energy density, and high flexibility compared to conventional hydraulic cylinders.

[0060] The beneficial effects of the present invention are:

[0061] ① The present invention provides an intelligent chassis system for tracked vehicles in hilly and mountainous areas based on environmental perception and road surface recognition, which is not only suitable for field operations of tracked vehicles in complex hilly and mountainous areas, but also suitable for transfer driving in working environments of tracked vehicles;

[0062] ② The environment perception and road recognition algorithm provided by the present invention realizes multi-modal perception and fusion decision-making of the mobile terminal, and innovatively combines the FPN+ResNet50 algorithm, IPM image inverse perspective algorithm, 3D sparse convolutional graph neural network, TSDF truncated signature distance algorithm, MDU algorithm, gated modality perception convolutional network and Pi-Sigma fuzzy neural network learning algorithm, while ensuring recognition accuracy and reliable decision-making, reducing the amount of calculation and computing cost, improving recognition efficiency, thereby achieving more accurate, timely and reliable recognition perception and feedback control;

[0063] ③ The intelligent chassis structure provided by the present invention can adjust the track tension and track inclination angle in real time, greatly improving the stability and passability of tracked vehicles under various complex road conditions, while effectively reducing the wear and damage of the track, improving the reliability of tracked vehicles, and expanding the application scenarios and scope of tracked vehicles;

[0064] ④ The present invention helps realize the automated operation and automatic driving of tracked vehicles, improves the efficiency and safety of field operations and transfer, saves manpower and material resources, and has broad market prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is the whole machine structure diagram of the present invention;

[0066] Figure 2 It is a top view of the whole machine of the present invention;

[0067] Figure 3 is a system control flow chart of the present invention;

[0068] Figure 4 It is the overall structural framework diagram of the present invention;

[0069] Figure 5 It is a framework diagram of the multimodal fusion environment perception system included in the present invention.

[0070] In the figure: 1-engine, 2-gearbox, 3-drive shaft, 4-track, 5-drive wheel, 6-guide wheel, 7-tensioning wheel, 8-front auxiliary wheel, 9-No. 2 servo hydraulic cylinder, 10-front auxiliary wheel connecting rod, 11-front auxiliary wheel swing arm, 12-frame, 13-hydraulic oil pump, 14-No. 1 servo hydraulic cylinder, 15-tensioning wheel swing arm, 16-tensioning wheel connecting rod 16, 17-the figure shows the main control core of the edge computer JetsonXavier NX-T506S, 19-servo drive. DETAILED DESCRIPTION

[0071] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments are within the scope of protection of the present invention.

[0072] Example:

[0073] like Figure 1 As shown, the present invention provides an intelligent chassis system for tracked vehicles in hilly and mountainous areas based on environmental perception and road surface recognition, including a mechanical structure part and a control system part, wherein the mechanical structure part includes an engine 1, a gearbox 2, a transmission shaft 3, a track 4, a driving wheel 5, a guide wheel 6 (including a supporting wheel, a sprocket wheel, a guide wheel, etc.), a frame 12, a hydraulic oil pump 13, and a track tension adjustment device and a track inclination adjustment device. The track tension adjustment device includes a pair of tensioning wheels 7, a pair of tensioning wheel swing rods 15, a tensioning wheel connecting rod 16 and a No. 1 servo hydraulic cylinder 14, characterized in that the height of the tensioning wheel 7 is adjusted by the servo hydraulic cylinder to adjust the track tension. The track inclination adjustment device includes a pair of front auxiliary wheels 8, a pair of front auxiliary wheel swing rods 11, a front auxiliary wheel connecting rod 10 and a No. 2 servo hydraulic cylinder 9, characterized in that the front auxiliary wheel 8 is lifted by a hydraulic rod to adjust the track inclination.

[0074] The control system consists of three layers. The first layer is the perception layer, which includes two industrial CMOS vision cameras 20, a four-line mechanical laser radar 21, and a 6-axis IMU inertial sensor 22 to identify the terrain type and landform features that the tracked vehicle is traveling on and about to reach. The second layer is the control layer, which is composed of the edge computer Jetson Xavier NX-T506S main control core 17 as the main control core of the tracked vehicle, and the Robot Operating System (ROS) 18 as the main control system, responsible for image processing. The third layer is the drive layer, which mainly includes a servo driver 19 to receive host signals and send drive signals to the corresponding servo hydraulic cylinders.

[0075] The algorithm comprises the following steps:

[0076] Step (1), using the Jetson Xavier NX-T506S main control core 17 as the main control core of the tracked vehicle, the Robot Operating System as the main control system, the RK3588J as the main control board, two industrial CMOS vision cameras 20 to form a binocular vision system, a four-line mechanical laser radar 21 and a 6-axis IMU inertial sensor 22 as an environmental perception unit;

[0077] Step (2), when the tracked vehicle is moving, two industrial CMOS vision cameras 20, laser radar 21, and 6-axis IMU inertial sensor 22 are used to capture the 2D image, 3D point cloud information, and real-time posture information of the vehicle in front of the vehicle in real time, and transmit the obtained data back to the Jetson Xavier NX-T506S main control core 17;

[0078] Step (3), using the FPN+ResNet50 algorithm and IPM image inverse perspective transformation to perform multi-scale image feature extraction on the two-dimensional visual image, and using the 3D sparse convolutional graph neural network and TSDF truncated signature distance algorithm to voxelize the three-dimensional point cloud;

[0079] Step (4), using progressive cascade connection, assisted by MDU and gated modality-aware convolutional network to obtain multimodal voxel features, and realize multimodal fusion interaction of camera image and lidar point cloud;

[0080] Step (5), using the Pi-Sigma fuzzy neural network learning algorithm to make a final decision on all the collected data to obtain the road type and terrain feature information ahead;

[0081] Step (6), the main control unit can determine whether the vehicle condition allows passing the obstacle ahead based on the road information obtained by identification. If it cannot pass, the vehicle stops moving forward and issues an alarm prompting manual processing and control. If it can pass, the main control unit selects the appropriate track tightness mode and forward inclination angle according to the road information, and sends the corresponding command signal to the servo driver 19;

[0082] Step (7), after receiving the signal, the servo driver 19 sends a signal to control the No. 1 servo hydraulic cylinder 14 and the No. 2 servo hydraulic cylinder 9, completes the adjustment and control of the tensioning wheel 7 and the front auxiliary wheel 8, and adjusts the chassis parameters to cope with the next road condition;

[0083] Step (8), the vehicle moves forward, repeating the above steps until it reaches the navigation destination, and the vehicle stops moving forward. If an impassable road appears during this period, or if an identification error causes abnormal adjustment of the height of the tensioning wheel 7 or the position of the front auxiliary wheel 8, it will be handled manually.

Claims

1. An intelligent chassis system for tracked vehicles in hilly and mountainous areas based on environmental perception and road surface recognition, characterized in that: Including mechanical structure part and control system part; The mechanical structure part comprises an engine (1), a gearbox (2), a transmission shaft (3), a crawler track (4), a driving wheel (5), a guide wheel (6), a frame (12), a hydraulic oil pump (13), and a crawler track tension adjustment device and a crawler track inclination adjustment device. The crawler track tension adjustment device comprises a pair of tensioning wheels (7), a pair of tensioning wheel swing rods (15), a tensioning wheel connecting rod (16) and a No. 1 servo hydraulic cylinder (14). The height of the tensioning wheel (7) is adjusted by the No. 1 servo hydraulic cylinder (14) to adjust the crawler track tension. The crawler track inclination adjustment device comprises a pair of front auxiliary wheels (8), a pair of front auxiliary wheel swing rods (11), a front auxiliary wheel connecting rod (10) and a No. 2 servo hydraulic cylinder (9). The front auxiliary wheel (8) is lifted by the No. 2 servo hydraulic cylinder (9) to adjust the crawler track inclination. The industrial CMOS visual camera (20) is two high-definition camera lens modules installed in front of the crawler chassis, the four-line mechanical laser radar (21) is installed on the top of the crawler vehicle, and the 6-axis IMU inertial sensor (22) is installed inside the chassis; A No. 1 servo hydraulic cylinder (14) is installed in the middle of the vehicle frame (12) through a bearing base, and its piston rod is connected to a tension wheel connecting rod (16). A pair of tension wheels (7) at both ends of the connecting rod are respectively installed on the upper inner side of the middle of the left and right crawlers (4). The tension wheel connecting rod (16) is connected to a pair of tension wheel swing rods (15) respectively located on both sides of the vehicle frame (12), forming a set of slider swing rod mechanism. When the No. 1 servo hydraulic cylinder (14) drives the piston rod to extend and retract, the tension wheels (7) on both sides synchronously complete the rising and falling actions, thereby ensuring that the tightness of the crawlers on both sides is the same. The second servo hydraulic cylinder (9) is installed at the front of the vehicle frame (12), and its piston rod is connected to the front auxiliary wheel connecting rod (10). A pair of front auxiliary wheels (8) at both ends of the connecting rod are respectively installed at the front upper inner side of the left and right crawlers (4). The front auxiliary wheel connecting rod (10) is connected to a pair of front auxiliary wheel swing rods (11) respectively located on both sides of the vehicle frame (12), forming a set of slider swing rod mechanism. When the second servo hydraulic cylinder (9) drives the piston rod to extend and retract, the front auxiliary wheels (8) on both sides synchronously complete the rising and falling actions, thereby ensuring that the front inclination angles of the crawlers on both sides are consistent; The control system consists of a three-layer structure. The first layer is the perception layer, which includes two industrial CMOS visual cameras (20), a four-line mechanical laser radar (21), and a 6-axis IMU inertial sensor (22) for identifying the type and topographic features of the terrain that the tracked vehicle is traveling on and about to reach. The second layer is the control layer, which is composed of the edge computer Jetson Xavier NX-T506S main control core (17) as the main control core of the tracked vehicle, and the Robot Operating System (18) as the main control system responsible for image processing; the third layer is the drive layer, including the servo driver (19) for receiving the host signal and sending the drive signal to the corresponding servo hydraulic cylinder.

2. The system according to claim 1, characterized in that: The control system includes an environment perception module and a numerical control hydraulic module, wherein the environment perception module is used to obtain images and point cloud information obtained by an industrial CMOS visual camera (20) and a four-line mechanical laser radar (21), and transmit the collected image data back to the edge computer Jetson Xavier NX-T506S main control core (17) shown in the figure, and generate control information based on the judgment result. The numerical control hydraulic module uses the control information generated based on the Jetson Xavier NX-T506S main control core (17) to accurately control the stroke of the first servo hydraulic cylinder (14) and the second servo hydraulic cylinder (9), thereby accurately controlling the height of the tension wheel (7) and the position of the front auxiliary wheel (8). During this period, if the system crashes or the recognition error causes the height of the tension wheel (7) or the position of the front auxiliary wheel (8) to be abnormally adjusted, it is handed over to manual remote control to improve the robustness of the system.

3. An algorithm based on environment perception and road surface recognition, comprising the following steps: Step (1), when the tracked vehicle is moving, two industrial CMOS visual cameras (20) are used to collect visual images of the road in the forward direction, a laser radar (21) is used to scan the road in the forward direction to establish point cloud information, and a 6-axis IMU inertial sensor (22) is used to record the vehicle acceleration, inclination, impact vibration, and rotation posture information in real time, and the obtained data is transmitted back to the Jetson XavierNX-T506S main control core (17); Step (2), for the visual image captured by the industrial camera (20), after extracting multi-scale image features using the FPN+ResNet50 algorithm, the image is mapped from the image coordinate system to the world coordinate system using the IPM image inverse perspective transformation, thereby eliminating the interference and error that the perspective effect may cause to subsequent image processing operations, and finally obtaining the position coordinates of the extracted features; the formula is as follows, wherein 2α represents the camera image field of view, m and n represent the height and width of the image, h represents the height offset of the camera relative to the object, and u i 、v i Represents the pixel index, x p ,y p Represents image u i 、v i The corresponding estimated distance in the vehicle coordinate system: For the point cloud information obtained by laser radar (21) scanning, a set of 3D sparse convolutional graph neural networks is used to convert the point cloud into voxels and extract their features in the voxel space. The 3D sparse convolutional graph neural network formula is expressed as follows, where x is the feature representation of each node i, y is the label corresponding to the node, A is the adjacency matrix representing the graph, W is the weight matrix, and H is the defined function: Each convolution block outputs voxel features at different scales, representing different levels of abstraction of the point cloud. The multi-scale features are further interacted with their respective 2D counterparts through the GMA convolution block. The TSDF truncated signature distance algorithm is used to calculate the TSDF value and RGB grayscale value of each voxel. The coordinate of voxel x on the TSDF map is recorded as (V x ,V y ,V z ), then the position of voxel x in physical world coordinates is: P x,wrd =(x0+v x ·voxel.x,y0+v y ·voxel.y,z0+v z ·voxel.z) (5) Calculate the position of voxel x in the camera coordinate system. Assume that the pose relative to the physical coordinate system is R and T. Then the position of voxel x in the camera coordinate system is: P x,cam =R·P x,wrd +T (6) Calculate the depth cam of voxel x relative to the camera according to the camera imaging model z (x), where K is the camera intrinsic parameter matrix, I x Represents the pixel coordinate of the voxel x projection under the camera plane: The intersection of a straight line drawn along the camera optical center and voxel x with the object cross section is recorded as point P, and the depth is recorded as d p , let the current depth map be D, and take d in the actual calculation p =D(I x ), then the sdf value of voxel x is calculated as follows: sdf(x)=d p -d x =D(I x )-cam z (x) (8) The tsdf value of voxel x can be calculated based on its sdf value. The formula is as follows: Traverse all voxels and calculate the TSDF weight of the current frame. The formula is as follows, where θ is the angle between the projection light and the surface normal vector: The current frame is re-fused with the previous fusion result, and the update is continuously iterated. The update of each voxel is based on the previous voxel. After weighted averaging according to certain rules, the obtained results are spliced ​​to obtain the final 3D model, where TSDF(p) is the fused TSDF value of voxel p, W(p) is the fusion weight value; tsdf(p) is the current TSDF value of voxel p, w(p) is the current frame weight value, and the TSDF value calculation and update formula is as follows: W(p)=W(p)+w(p) (12) Step (3) is to perform multimodal fusion of the geometric features and semantic features obtained above into the same scene, estimate the depth of the pixel, and then use its semantics of the 3D world to fuse with the real 3D point cloud. Each image feature pixel is regarded as a seed and a virtual point is generated in the BEV space. At the same time, the MDU algorithm is used to improve the geometric accuracy and semantic richness of the generated virtual points. d Represents the depth perception semantic feature, the formula is as follows: At the same time, under the guidance of the reference lidar point cloud voxels, a gated modality-aware convolutional network is used to control and update its camera corresponding target, adaptively select more valuable information parts from the camera voxel features, and then group them for sparse convolution to aggregate them to achieve full multimodal interaction. This gating formula is expressed as, where L represents a single radar voxel and C represents a single camera voxel: After executing MDU and gated modality-aware convolutional networks at different scales in the voxel space, corresponding multimodal voxel features are generated at each scale, denoted as F, and then progressive cascade connections across different scales are used to further aggregate these multi-granularity information. The formula is as follows: Through the above series of feature extraction and fusion, the multimodal fusion interaction of camera images and lidar point clouds is realized, and the final multimodal voxel features are sent to the recognition prediction module of the main control core as the basis for subsequent road type recognition and judgment; Step (4) compares the multimodal voxel features with the pre-trained common road model data set in hilly and mountainous areas. At the same time, the real-time vehicle dynamic information in the same frame on the time axis is referred to, and the Pi-Sigma fuzzy neural network learning algorithm is used for decision control, which is more adaptable to the dynamic characteristics of uncertain road systems and has stronger robustness and fault tolerance. The multimodal voxel features are used as input signals from the input layer to the fuzzification layer, where the input layer is 3 neurons, and each input is fuzzified using 5 membership functions. The formula is as follows, where x represents the neuron, c represents the membership function, i = 1, 2, 3, j = 1, 2, 3, 4, 5: The fuzzified data is processed in the fuzzy reasoning layer, and the 125 fuzzy rules formed by formula (9) are reasoned and output. The reasoning formula of each rule premise is as follows: The 125 fuzzy rule reasoning outputs formed by formula (9) are as follows: Finally, if the output layer is defuzzified, the final output is obtained. The formula is as follows: If the output layer gets the expected output, the learning algorithm ends. Otherwise, the gradient descent method is used to adjust the weight values ​​between the layers. The error e between the lth output and the corresponding ideal output in the neural network is l , the error performance index E of the pth sample p The function calculation formula is as follows, where L = 1 and N = 2 is the number of network output layers: The adjustment formula of the weight of the output layer is as follows, where i = 0, 1, 2, 3, j = 1, 2, 3, 4, 5, L = 1, 2, and x0 = 1 is taken as a fixed bias: Finally, the weight learning algorithm of the output layer is obtained, and the formula is as follows, where η is the learning rate and α is the momentum factor: For each iteration of multiple samples, each sample is trained and the weights are updated. After all samples are trained, the next iteration is performed until the requirements are met. The final decision is made on the road type and distance, as well as the width of the gully or the height of the ridge ahead. This result is sent to the control module in the main control core as a basis for subsequent mode selection. Step (5), after the control module obtains the road type obtained by decision, it selects the track tension mode according to the preset table and sends the corresponding command signal to the servo driver (19), which then receives the command and sends the corresponding control signal to the first servo hydraulic cylinder (14). If a ridge / gully appears in front, the calculated ridge height or gully width is compared with the vehicle parameters. If the vehicle cannot pass through, an alarm prompt is given to let the vehicle take over by remote control. If the vehicle can pass through, an extension command is sent to the servo driver (19). At the same time, in order to maintain the tension wheel height required for the same tension, a corresponding command signal is sent to the servo driver (19), which then receives the command and sends the corresponding control signal to the first servo hydraulic cylinder (14) and the second servo hydraulic cylinder (9). After passing through the ridge, a reset signal is sent. Step (6), in the tension adjustment mode, the No. 1 servo hydraulic cylinder (14) performs the actions of lifting, lowering and locking the tension wheel connecting rod (16) according to the received servo signal, driving the tension wheels (7) at both ends to rise and fall, thereby adjusting the track tension. In the caster angle adjustment mode, the No. 2 servo hydraulic cylinder (9) performs the actions of pushing, pulling and locking the front auxiliary wheel connecting rod (10) according to the received servo signal, driving the front auxiliary wheels (8) at both ends to rise and fall, thereby adjusting the track caster angle. At the same time, the No. 1 servo hydraulic cylinder (14) synchronously lifts, lowers and locks according to the signal, ensuring that the tension of the track remains unchanged while the position of the front auxiliary wheel (8) changes. The tensioning force formula required for the locking action is as follows, wherein W is the load weight of the track, F represents the resistance, V represents the track speed (m / s), and μ represents the friction coefficient: Step (7), the vehicle continues to travel, repeating the above steps (1) to (6) until the tracked vehicle reaches the destination. During this period, if an impassable road appears, or if an identification error causes abnormal adjustment of the height of the tensioning wheel (7) or the position of the front auxiliary wheel (8), the vehicle stops and an alarm is sounded, and the vehicle is handed over to manual remote control for processing, thereby improving the robustness of the system.

Citation Information

Cited By

  • Man-machine interaction method and system for mechanical arm

    CN119897858A

  • Mechanical arm human-computer interaction method and system

    CN119897858B

  • Multi-target detection method

    CN120387144A

  • Binocular vision road width measurement method and system for vehicle-road cloud cooperation

    CN121600086A