Self-adaptive terrain load moving vehicle based on multi-mode AI navigation and cooperative control system of self-adaptive terrain load moving vehicle

Abstract

The invention belongs to the field of article carrying, and particularly relates to a self-adaptive terrain load moving vehicle based on multi-mode AI navigation, which comprises a crawler-type chassis driving mechanism, a lock catch arranged on the outer side wall of a chassis main body, crawler main bodies arranged on the two sides of the chassis main body, a plurality of groups of gears arranged in the crawler main bodies, and a plurality of groups of driving mechanisms arranged on the crawler main bodies, through organic combination of infrared following, pattern recognition and the SLAM technology, full-autonomous path planning and dynamic obstacle avoidance of the load moving vehicle are achieved, manual intervention is completely eliminated, the automation level and efficiency of operation are greatly improved, and the load moving vehicle is suitable for complex and changeable operation environments. The terrain sensing system composed of the crawler chassis and the pressure sensor array is matched with the AI terrain surveying and mapping technology, complex terrains such as slopes, pits and non-hardened grounds can be accurately recognized and adapted, the space and terrain limitation of traditional weight moving equipment is effectively broken through, and the operation range is widened.

Description

Technical Field

[0001] The present invention belongs to the field of article handling, and more specifically to an adaptive terrain-adaptive load-shifting vehicle based on multimodal AI navigation and its collaborative control system. Background Art

[0002] In modern industrial production, warehousing and logistics, engineering construction, and other fields, the efficient and safe movement of heavy objects is a key component in ensuring smooth operational processes. Currently, commonly used heavy-load moving equipment in the industry includes manual forklifts, overhead cranes, remote-controlled electric load-transfer vehicles, and manual load-transfer vehicles. However, these traditional methods all have significant limitations in practical applications. Manual forklifts rely on professional drivers for full control, and prolonged operation can easily lead to fatigue. This not only makes 24 / 7 operation difficult, resulting in high labor costs, but also carries the risk of operator error, such as cargo collisions or personal injuries caused by human judgment errors. Driver safety is particularly difficult to guarantee in hazardous environments such as high temperatures, dust, and toxic substances. Furthermore, handling speed and accuracy are limited by the driver's proficiency, and fatigue during repetitive operations can easily lead to reduced accuracy, making them unable to meet the demands of large-scale, high-intensity tasks.

[0003] While cranes are suitable for lifting heavy objects in specific scenarios, their reliance on fixed tracks or overhead structures severely limits their operational flexibility, making them unsuitable for use in confined spaces or complex terrain. Furthermore, the lifting and positioning process requires collaboration from multiple personnel, resulting in a cumbersome and time-consuming operation that demands extremely high operator skills. Furthermore, their lack of dynamic environmental awareness makes it difficult to avoid moving obstacles in real time, posing a significant safety hazard. While remote-controlled electric heavy-lift trucks reduce human interaction to some extent, they still require real-time operator control and lack fully autonomous path planning and dynamic obstacle avoidance. Remote control signals are prone to delays or errors during extended operations. Furthermore, their reliance on preset routes or simple sensors makes them difficult to navigate dynamic obstacles (such as pedestrians and temporary storage) and complex terrain (potholes and slopes). Furthermore, these devices lack intelligent features like mapping and task tracking, failing to meet the demands of modern logistics for data-driven and intelligent management. Manually pushing heavy vehicles relies entirely on manual labor, which is labor-intensive and only suitable for short-distance, light-load scenarios. They are not suitable for carrying oversized or heavy cargo (typically ≤ 1 ton). Furthermore, they lack automatic obstacle avoidance mechanisms, making them prone to tipping over on uneven surfaces or when the path deviates, resulting in lower safety and efficiency.

[0004] To this end, the present invention provides an adaptive terrain-shifting vehicle based on multimodal AI navigation and a collaborative control system thereof. Summary of the Invention

[0005] In order to make up for the deficiencies of the prior art, at least one technical problem raised in the background technology is solved.

[0006] The technical solution adopted by the present invention to solve its technical problems is as follows: the adaptive terrain-shifting vehicle based on multimodal AI navigation described in the present invention includes a crawler chassis drive mechanism, a multimodal sensing module, a power and energy module, a human-computer interaction module, a handling actuator and a map generation unit. The crawler chassis drive mechanism includes a chassis body, a locking buckle is provided on the outer wall of the chassis body, crawler bodies are provided on both sides of the chassis body, multiple sets of gears are provided inside the crawler body, a motor body is installed inside the chassis body, a reinforcement frame is built into the chassis body, and standardized mechanical interfaces are provided at the front and rear ends of the chassis body; six sets of embedded pressure sensor arrays are respectively installed on the crawler bodies on both sides of the chassis body for real-time monitoring of ground pressure distribution; The multimodal sensing module includes an infrared sensor, an infrared receiver, an ultrasonic sensor, a binocular camera, a lidar, and a cliff sensor. Infrared sensors are installed at the rear end and both sides of the chassis body, and the infrared receiver is integrated into the front end of the vehicle body to receive infrared signals and track ground infrared points. The ultrasonic sensor has a detection range of 0.2m-5m. The binocular camera is used for AI image recognition of obstacles and potholes. A lidar is installed on the top of the chassis body, and the lidar builds an environmental map based on the SLAM algorithm. A scanner is set in the middle of the bottom of the chassis body, and cliff sensors are set at both ends of the scanner. Speakers are set on both sides of one cliff sensor. The cliff sensor obtains the size of the transported object and the bottom support point data through the laser ranging module. Collision sensors are set on both sides of the front end of the chassis body. The power and energy module includes a dual-motor drive unit and an intelligent power management unit. The two motors independently control the crawler bodies on both sides to achieve differential steering and rotation on the spot. The intelligent power management unit is equipped with a lithium battery pack and a low-power chipset, and supports automatic recharging through RFID tag positioning. The human-computer interaction module includes a voice prompt unit and a mobile phone communication interface. The voice prompt unit triggers a safety warning voice when it detects people nearby. The mobile phone communication interface is connected to the mobile terminal through the operator's customized IoT card. The transport actuator includes a rotatable pallet equipped with a pressure sensor to prevent overloading. The bottom scanning data calibrates the pallet height and angle in real time. The rotatable pallet includes a bracket and a universal ball bearing bracket. Manual locks are set on both sides of the bracket for installation on the chassis body. The top of the bracket is equipped with a universal ball bearing bracket. The map generation unit, based on the SLAM algorithm, integrates inertial navigation and lidar data to generate high-precision two-dimensional terrain maps.

[0007] Preferably, the chassis adopts an expandable track chassis. When carrying heavy objects, the system automatically adjusts the track tension and motor torque distribution according to the pressure sensor array data.

[0008] Preferably, the multimodal sensing module also includes a 360° surround-view binocular camera group, a millimeter-wave radar, a ToF laser array and a structured light scanning module. The 360° surround-view binocular camera group consists of four groups of 200° wide-angle cameras, which combine the millimeter-wave radar and the ToF laser array to build a three-dimensional environmental model; the structured light scanning module scans the bottom contour of the transported object through a 100Hz high-frequency pulse to generate three-dimensional point cloud data.

[0009] The collaborative control system for adaptive terrain-adaptive heavy-duty vehicles based on multimodal AI navigation includes: The motion control logic unit has both infrared following mode and remote control following mode. The infrared following mode uses an infrared receiver to analyze the coordinates of the light spot and uses the PID algorithm to dynamically adjust the track speed to maintain a relative distance error of ≤5cm from the target point. The remote control following mode uses UWB positioning technology to calculate the azimuth and distance between the remote control and the vehicle in real time, and combines the AI ​​obstacle avoidance algorithm to plan the following path. The intelligent navigation system unit includes map construction and path planning modules. Map construction uses raster maps to store terrain information, uses the A* shortest path algorithm to generate a global path, and uses the DWA algorithm to achieve local dynamic obstacle avoidance. When the user clicks the target point through the mobile phone interface, the system automatically calls historical map data to calculate the optimal route. The safety control strategy unit includes a handling calibration module and a human-machine collaborative safety module. The handling calibration module uses a bottom scanner to monitor the center of gravity offset of the cargo in real time and controls the pallet's posture through PID to ensure handling stability and a tilt angle of ≤3°. The human-machine collaborative safety module controls the load-transporting vehicle to slow down to 0.2m / s and continuously broadcasts a prompt when the infrared thermal imaging detects a human body entering the 1m range. The data communication architecture unit uses the MQTT communication protocol to achieve data synchronization between the vehicle and the mobile phone, encrypted transmission of map data, and supports offline task caching and breakpoint resumption functions.

[0010] Preferably, the obstacle avoidance decision module of the motion control logic unit fuses the data of the ultrasonic sensor, binocular camera and lidar, classifies the obstacle type through a convolutional neural network, and adopts a priority avoidance strategy.

[0011] Preferably, the intelligent navigation system unit also has an automatic recharging mechanism. By pre-storing the charging room coordinates and real-time positioning data, it triggers the return instruction to enable the energy-saving mode and turn off non-essential loads.

[0012] Preferably, the safety control strategy unit also includes a four-level warning mechanism: when a moving obstacle is detected 3 meters away, the LED warning light flashes; When a static obstacle is detected within 1.5 meters, an audible and visual alarm is activated; In the 0.5m emergency braking zone, the power output is cut off and the electromagnetic brake is activated; When the tilt angle of the mobile truck is greater than 15°, the hydraulic suspension is automatically locked to prevent rollover.

[0013] Preferably, the system upgrade emergency obstacle avoidance instructions are transmitted via LoRa to ensure that emergency response is completed within 20ms; the intelligent path planning algorithm introduces a spatiotemporal joint optimization model, based on the real-time map constructed by SLAM, integrates historical terrain data, and dynamically adjusts the algorithm weight coefficient to achieve the optimal balance between path length and safety distance.

[0014] The beneficial effects of the present invention are as follows: 1. The adaptive terrain-adaptive load-shifting vehicle based on multimodal AI navigation and its collaborative control system described in the present invention, through the organic combination of infrared tracking, pattern recognition and SLAM technology, enables the load-shifting vehicle to achieve fully autonomous path planning and dynamic obstacle avoidance, completely eliminating human intervention, significantly improving the level of operation automation and efficiency, and is suitable for complex and changing working environments.

[0015] 2. The adaptive terrain-adaptive load-shifting vehicle based on multimodal AI navigation and its coordinated control system, as well as the terrain perception system consisting of a tracked chassis and a pressure sensor array, combined with AI terrain mapping technology, can accurately identify and adapt to complex terrain such as slopes, potholes, and unhardened surfaces, effectively breaking through the spatial and terrain limitations of traditional load-shifting equipment and expanding its operating range.

[0016] 3. The adaptive terrain-adapting load-shifting vehicle based on multimodal AI navigation and its coordinated control system described in this invention support 24-hour uninterrupted operation. Combined with automatic recharging and task queue management functions, they significantly reduce manpower dependence and operating costs. The equipment has a load capacity of 3-5 tons and is compact in size, achieving a load ratio 3-5 times that of traditional equipment in narrow spaces, greatly improving the load-bearing efficiency per unit space.

[0017] 4. The adaptive terrain-shifting vehicle based on multimodal AI navigation and its coordinated control system described in this invention uses millimeter-wave radar and an AI vision system to monitor the surrounding environment in real time. Once an obstacle or approaching person is detected, path calibration and voice warnings are immediately triggered. Combined with a collision prediction mechanism, this builds a comprehensive, multi-layered safety net to effectively reduce operational risks.

[0018] 5. The adaptive terrain-adaptive load-shifting vehicle based on multimodal AI navigation and its collaborative control system described in the present invention automatically generates high-precision terrain maps during operation, supports visual operation on mobile phones, and realizes convenient equipment scheduling; at the same time, the system records the handling path and load data in detail, providing a scientific basis for optimizing the operation process, supporting the collaborative scheduling of multiple devices, realizing efficient resource utilization and improving overall operation efficiency.

[0019] 6. The adaptive terrain-adaptive load-shifting vehicle based on multimodal AI navigation and its collaborative control system described in the present invention adopt a modular design, and core components can be upgraded independently without replacing the entire equipment, significantly reducing long-term maintenance costs. The equipment structure is highly durable and, combined with an intelligent fault diagnosis system, reduces maintenance frequency and further enhances the value of the equipment over its entire life cycle. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The present invention will be further described below with reference to the accompanying drawings.

[0021] Figure 1 It is a flow chart of the present invention; Figure 2 This is a side structural diagram of the chassis in the present invention; Figure 3 This is a structural diagram of the chassis top in the present invention; Figure 4 This is a diagram of the bottom structure of the chassis in the present invention; Figure 5 This is a front view structural diagram of the tray in the present invention; Figure 6 This is a top view of the structure of the tray in the present invention; Figure 7 This is the front view structure of the forklift module in the present invention Figure 1 ; Figure 8 This is the front view structure of the forklift module in the present invention Figure 2 ; Figure 9 This is a front view structural diagram of the box transportation module in the present invention.

[0022] In the figure: 1. Chassis body; 2. Lock; 3. Infrared sensor; 31. Infrared receiver; 4. LiDAR; 5. Track body; 6. Gear; 7. Collision sensor; 8. Motor body; 9. Scanner; 91. Speaker; 92. Cliff sensor; 10. Bracket; 101. Universal ball tray. DETAILED DESCRIPTION

[0023] In order to make the technical means, creative features, objectives and effects achieved by the present invention easier to understand, the present invention is further described below in conjunction with specific implementation methods.

[0024] like Figures 1 to 9As shown, the adaptive terrain-shifting heavy vehicle based on multimodal AI navigation according to an embodiment of the present invention includes a crawler chassis drive mechanism, a multimodal sensing module, a power and energy module, a human-computer interaction module, a handling actuator and a map generation unit. The crawler chassis drive mechanism includes a chassis body 1, a lock buckle 2 is provided on the outer wall of the chassis body 1, crawler bodies 5 are provided on both sides of the chassis body 1, multiple sets of gears 6 are provided inside the crawler body 5, a motor body 8 is installed inside the chassis body 1, a reinforcement frame is built into the chassis body 1, and standardized mechanical interfaces are provided at the front and rear ends of the chassis body 1; six sets of embedded pressure sensor arrays are respectively installed on the crawler bodies 5 on both sides of the chassis body 1 for real-time monitoring of ground pressure distribution; The multimodal sensing module includes an infrared sensor 3, an infrared receiver 31, an ultrasonic sensor, a binocular camera, a laser radar 4, and a cliff sensor 92. The infrared sensor 3 is installed at the rear end and both sides of the chassis body 1. The infrared receiver 31 is integrated into the front end of the vehicle body for receiving infrared signals and ground infrared point tracking; the ultrasonic sensor has a detection range of 0.2m-5m; the binocular camera is used for AI image recognition of obstacles and potholes; a laser radar 4 is installed on the top of the chassis body 1, and the laser radar 4 builds an environmental map based on the SLAM algorithm; a scanner 9 is provided in the middle position of the bottom of the chassis body 1, and cliff sensors 92 are provided at both ends of the scanner 9, and speakers 91 are provided on both sides of one of the cliff sensors 92. The cliff sensor 92 obtains the size of the transported object and the bottom support point data through the laser ranging module; collision sensors 7 are provided on both sides of the front end of the chassis body 1; The power and energy module includes a dual-motor drive unit and an intelligent power management unit. The two motor bodies 8 independently control the crawler bodies 5 on both sides to achieve differential steering and rotation on the spot. The intelligent power management unit is equipped with a lithium battery pack and a low-power chipset, and supports automatic recharging through RFID tag positioning. The human-computer interaction module includes a voice prompt unit and a mobile phone communication interface. The voice prompt unit triggers a safety warning voice when it detects people nearby. The mobile phone communication interface is connected to the mobile terminal through the operator's customized IoT card. The transport actuator includes a rotatable tray equipped with a pressure sensor to prevent overloading. The bottom scanning data is used to calibrate the tray height and angle in real time. The rotatable tray includes a bracket 10 and a universal ball tray 101. Manual locks are provided on both sides of the bracket 10 for installation on the chassis body 1. The universal ball tray 101 is provided on the top of the bracket 10. The map generation unit, based on the SLAM algorithm, integrates inertial navigation and LiDAR 4 data to generate high-precision two-dimensional terrain maps.

[0025] The chassis adopts an expandable track chassis. When carrying heavy objects, the system automatically adjusts the track tension and motor torque distribution according to the pressure sensor array data.

[0026] The multimodal sensing module also includes a 360-degree surround-view binocular camera group, a millimeter-wave radar, a ToF laser array, and a structured light scanning module. The 360-degree surround-view binocular camera group consists of four groups of 200-degree wide-angle cameras, which combine with the millimeter-wave radar and ToF laser array to build a three-dimensional environmental model; the structured light scanning module uses 100Hz high-frequency pulses to scan the bottom contour of the transported object to generate three-dimensional point cloud data.

[0027] The collaborative control system for adaptive terrain-adaptive heavy-duty vehicles based on multimodal AI navigation includes: The motion control logic unit has both infrared following mode and remote control following mode. The infrared following mode uses the infrared receiver 31 to analyze the coordinates of the light spot and dynamically adjusts the track speed using the PID algorithm to maintain a relative distance error of ≤5cm from the target point. The remote control following mode uses UWB positioning technology to calculate the azimuth and distance between the remote control and the vehicle in real time, and combines the AI ​​obstacle avoidance algorithm to plan the following path. The intelligent navigation system unit includes map construction and path planning modules. Map construction uses raster maps to store terrain information, uses the A* shortest path algorithm to generate a global path, and uses the DWA algorithm to achieve local dynamic obstacle avoidance. When the user clicks the target point through the mobile phone interface, the system automatically calls historical map data to calculate the optimal route. The safety control strategy unit includes a handling calibration module and a human-machine collaborative safety module. The handling calibration module uses a bottom scanner 9 to monitor the center of gravity offset of the cargo in real time and controls the pallet's posture through PID to ensure handling stability and a tilt angle of ≤3°. The human-machine collaborative safety module controls the load-transporting vehicle to slow down to 0.2m / s and continuously broadcasts a prompt when the infrared thermal imaging detects a human body entering the 1m range. The data communication architecture unit uses the MQTT communication protocol to achieve data synchronization between the vehicle and the mobile phone, encrypted transmission of map data, and supports offline task caching and breakpoint resumption functions.

[0028] The obstacle avoidance decision module of the motion control logic unit fuses the data from the ultrasonic sensor, binocular camera and lidar 4, classifies the obstacle type through a convolutional neural network, and adopts a priority avoidance strategy.

[0029] The intelligent navigation system unit also has an automatic recharging mechanism. By pre-storing the charging room coordinates and real-time positioning data, it triggers the return command to enable energy-saving mode and turn off non-essential loads.

[0030] The safety control strategy unit also includes a four-level early warning mechanism: when a moving obstacle is detected 3 meters away, the LED warning light flashes; When a static obstacle is detected within 1.5 meters, an audible and visual alarm is activated; In the 0.5m emergency braking zone, the power output is cut off and the electromagnetic brake is activated; When the tilt angle of the mobile truck is greater than 15°, the hydraulic suspension is automatically locked to prevent rollover.

[0031] The system upgrade's emergency obstacle avoidance commands are transmitted via LoRa, ensuring emergency response within 20ms. The intelligent path planning algorithm introduces a spatiotemporal joint optimization model, based on a real-time map constructed by SLAM, integrating historical terrain data, and dynamically adjusting the algorithm's weight coefficients to achieve an optimal balance between path length and safety distance.

[0032] Specifically, 1. Requirements Analysis and System Architecture Design 1. Terrain complexity analysis Objective: To establish a complex terrain parameter system that can be quantitatively evaluated to provide data support for mechanical structure design and control algorithms. Slope angle: Use IMU to measure typical scenarios (warehouse / construction site / climbing); Obstacle density: LiDAR 4-point cloud cluster analysis to count the number of obstacles per unit area; Terrain relief: The structured light scanner 9 obtains the standard deviation of ground elevation.

[0033] ‌Scene classification table‌: Terrain Level slope Obstacle density Application Scenario L1 (simple) ≤5° ≤0.5 Warehouse Lawn L2 (medium) 5°~15° 0.5~2 construction site L3 (complex) 15°~25° 2~5 Road climbing Hardware Design‌: Track selection: L1-L3 terrain uses metal articulated tracks; Sensor configuration: Binocular vision + LiDAR 4 fusion perception is mandatory for terrains above L2; Power redundancy: Use capacitor modules to handle instantaneous high currents.

[0034] 2. Software Control Hierarchical Analysis ‌Objective‌: Build a mode switching mechanism with priority preemption capability to ensure system security and real-time performance; Core Design Status hierarchy (top-down priority): [Emergency Brake]-->A[Safety Layer] [Obstacle Avoidance]-->B[Navigation Layer] [Path Following]-->B[Navigation Layer] [Charging Mode]-->B[Navigation Layer] [Manual remote control]-->B[Navigation layer] Safety layer A (highest priority): Thermal imaging human body detection / exceeding tilt angle limit / battery over-discharge; B Navigation layer (normal operation): autonomous navigation / path planning / multi-vehicle collaboration.

[0035] ‌Mode switching trigger conditions‌: Current Mode Trigger Event Target Mode Path Following Ultrasonic detection obstacle <1m Obstacle avoidance and detour Any mode IMU detects tilt > 5°, a person is closer than 1m, and the battery temperature is too high emergency braking Charging mode Manual remote control signal input Automatic recharge Mode switching verification method: Formal verification of critical paths (e.g., whether emergency braking can be effective within three control cycles); Monte Carlo simulation tests 5000 mode switching scenarios.

[0036] 3. Communication layer analysis ‌Goal‌: MQTT+LoRa hybrid protocol, building a communication network that balances real-time performance and wide coverage; ‌Protocol stack architecture.‌ Application layer: MQTT (message broker); transport layer: TCP / UDP; network layer: LoRaWAN.

[0037] Dual-band division of labor strategy: MQTT over Wi-Fi‌: Transmit high-bandwidth data: LiDAR 4 point cloud; real-time control instructions: motor control messages.

[0038] LoRa: Transmit low-rate data: GPS coordinates; emergency command broadcast: emergency stop signal (occupying the channel). Hybrid topology design: Main control device – signal -> AP [vehicle AP]; AP--LoRa-->operating vehicle; AP--4G-->Cloud[cloud platform]; Cloud--MQTT-->Mobile[Mobile terminal].

[0039] The master control device serves as a mobile base station, equipped with a dual-frequency signal transceiver, and adopts the TDMA time division multiple access mechanism: every 200ms is divided into 10 time slots, and control instructions are allocated first.

[0040] Data packet structure design: Frame header; Vehicle ID; Battery voltage; Emergency code: such as emergency stop, capsize warning; Check code; QoS guarantee mechanism: Set up a three-stage reaction: QoS level Number of retransmissions Application Scenario 0 0 General status data 1 3 Control instructions 2 5+ confirmation frames Emergency Order 4. Post-reliability test analysis Hardware-in-the-Loop (HIL) testing: Use NIVeriStand to simulate terrain parameter input and monitor the control board CAN bus response time.

[0041] Communication stress test: Use Cisco IOL to simulate multiple devices concurrently and calculate the packet loss rate and delay distribution.

[0042] Security Boundary Testing: Use fault injection tools (such as JAUS) to simulate sensor failures and verify whether the state machine can be degraded to a minimally safe mode.

[0043] 2. Hardware Basic Platform Design 1. Mechanical system design (1) Sensor array layout optimization Design goal: To achieve accurate measurement of track ground pressure and dynamic load distribution.

[0044] ‌Implementation steps‌: Sensor Selection and Layout: MEMS piezoresistive sensors are used; six measurement points are set up on each track side, in a staggered layout, installed in the internal cavity of the track shoe and connected via flexible circuits. Data exchange is carried out via a battery-powered wireless module.

[0045] (2) Standardization of scalable interfaces Design Specifications: Interface Type parameter Electrical parameters Power interface 24V Maximum current 30A CAN bus Serial communication protocol Baud rate 1Mbps Expansion IO Expansion chip Optoelectronic isolation 24V / 100mA Mechanical lock SAEJ1926-1 standard Tensile strength ≥500N Implementation: Use IP67 waterproof connector and configure reverse voltage protection circuit.

[0046] 2. Chassis and handling mechanism Cliff Sensor 92: Two sets of cliff sensors (92) are located at each rear end of the chassis. These use infrared Time-of-Flight (TOF) technology to detect height differences (such as potholes and steps) ahead in real time. The detection range is 0.1-1.5 meters, with an accuracy of ±2 cm. If a height difference greater than 10 cm is detected, emergency braking is triggered and the vehicle retreats to a safe area to prevent falls.

[0047] Omnidirectional pressure sensor: The pallet's top is integrated with an array of eight omnidirectional pressure sensors, made of flexible piezoresistive film, covering a 360-degree angle. When an obstruction (such as a low beam or hanging object) exerting a pressure of 5N or more is detected, the lift stops immediately and a voice alarm is triggered, preventing mechanical jams.

[0048] 3. Multimodal obstacle avoidance system (1) Visual sensor (binocular camera) Binocular Vision Module: Equipped with a global shutter binocular camera, it generates a depth map using a parallax algorithm. After fusing with LiDAR 4 data, it enhances the following functions: Stereo matching: Identify transparent / reflective obstacles (such as glass partitions) to fill the blind spots of LiDAR 4 detection; Semantic segmentation: Distinguish dynamic obstacles (pedestrians, vehicles) from static obstacles based on the YOLO model and optimize obstacle avoidance priority.

[0049] (2) Millimeter-wave radar Millimeter-wave radar: Installed at the four corners of the vehicle, with a detection range of 0.5m-50m and a speed resolution of 0.1m / s. It uses FMCW frequency modulation continuous wave technology to achieve the following functions: Dynamic target tracking: Calculate the relative speed and trajectory of moving obstacles in real time to predict collision risks (such as a forklift cutting in laterally); Adaptability to harsh environments: Replace optical sensors in rainy, foggy, and dusty environments to ensure safe operations in low-visibility scenarios.

[0050] (3) Optimization of ultrasonic sensors and infrared sensors Ultrasonic sensor: covers the near field of the vehicle body (0.2m-3m), adopts a multi-band alternating transmission strategy (40kHz / 58kHz), reduces multipath interference, and improves the reliability of dense obstacle detection; Infrared sensor: Added anti-ambient light interference algorithm to filter out sunlight interference by modulating and encoding infrared signals to ensure detection stability.

[0051] 4. Safety control enhancement sensors (1) Collision sensor 7 Redundant collision detection: In addition to laser / visual warning, piezoelectric collision sensors 7 (sensitivity adjustable range: 5N-50N) are installed around the vehicle body. The trigger conditions are divided into two levels: Touch alarm (5N-20N): reduce the speed to 0.1m / s and start local path replanning; Forced collision (>20N): Cut off the power and activate the electromagnetic brake, which needs to be reset manually.

[0052] (2) Gyroscope and accelerometer Attitude fusion perception: Using a six-axis IMU and fusing data through Kalman filtering, it achieves: Tilt angle compensation: Dynamically adjust the hydraulic suspension of the pallet to ensure that the tilt angle of the cargo is ≤3°; Fall warning: When the Z-axis acceleration is greater than 2g and lasts for 10ms, it is determined to be a fall risk and the drive wheels are immediately locked.

[0053] 3. Electronic system design 1. Spatiotemporal synchronization of LiDAR 4 and binocular camera Hardware synchronization solution: Generate synchronization pulses using FPGA: Software calibration process: Collect synchronous data of the checkerboard calibration plate; Calculate delay compensation: ; Coordinate system transformation matrix calibration: .

[0054] 2. Anti-vibration compensation Install the shock-absorbing bracket, use the IMU data preprocessing algorithm to estimate the vibration frequency based on the angular velocity around the Z axis, dynamically adjust the scanning frequency, perform coordinate correction, and then perform relevant compensation.

[0055] ‌‌Control logic‌: A[IMU data acquisition]-->B[Vibration frequency>20Hz] B--Yes-->C[Reduce scanning frequency to 60Hz] B--No-->D[Maintain 100Hz scanning] C-->E[Enable motion blur compensation] D-->F[Normal mode].

[0056] 3. Hardware synchronization mechanism between millimeter-wave radar and IMU: PPS pulse alignment: Send synchronous trigger signals to millimeter-wave radar, IMU, and binocular camera through FPGA, with time deviation less than 10ms; Dynamic calibration: Based on the Lie group SE(3) model, the extrinsic parameter matrix is ​​optimized online to compensate for the sensor pose offset caused by vehicle body vibration.

[0057] 4. Post-reliability test analysis 1. Track pressure distribution test Build a slope test platform (angle adjustable 0°-30°); Measure the pressure distribution under different loads: slope, load (kg), front pressure ratio, rear pressure ratio. ‌ 2. Synchronization accuracy verification Use a high-speed oscilloscope to measure the sensor trigger signal deviation, the maximum delay of single LiDAR 4 and the LiDAR + dual-eye radar.

[0058] 3. Vibration compensation effect Tested under 5Hz mechanical vibration: no compensation, hardware vibration reduction, and dynamic compensation corresponding to the point error and scanning success rate.

[0059] 5. Embedded System Development Solution 1. Main control unit selection and architecture design ‌‌Goal‌ : To achieve efficient processing of sensor data and real-time operation of navigation algorithms.

[0060] The FPGA chip processes the raw sensor data: the lidar 4-point cloud (200,000 points / second) and camera images (30fps), and has parallel computing capabilities: 16 DSP slices process data filtering.

[0061] The ARM‌ chip runs navigation algorithms (A* shortest global path algorithm, local path planning algorithm DWA) and real-time task processing: motor control, communication protocol stack.

[0062] Data interaction architecture: FPGA--AXI-Stream high-speed transmission protocol-->shared memory; ARM--DMA read-->shared memory; Sensor data-->FPGA pre-processing-->shared memory-->ARM algorithm processing; ‌Interrupt mechanism‌: FPGA triggers ARM interrupt response.‌ 2. Driver layer development (1) Motor closed-loop control protocol (CAN bus) Protocol design: The data message format consists of CAN ID and Data data, which records the motor ID, control mode (including speed and position), target value (speed, length), PID parameters, and checksum.

[0063] Control flow: Read encoder → Calculate error → PID calculation → Send CAN command.

[0064] (2) Sensor data preprocessing pipeline Point cloud sampling: Through voxel filtering parameters, FPGA accelerated voxel grid generation module realizes image distortion correction. 3. Post-reliability test analysis ‌Test modules: FPGA preprocessing delay, ARM algorithm cycle, CAN bus response, image correction delay.

[0065] (1) Real-time test: LatencyTop tool monitoring‌ .

[0066] (2) Hardware-in-the-loop testing: NIVeriStand simulation scenario‌.

[0067] VI. AI Algorithm Technology Implementation Plan 1. Computing platform construction (1) Processor and radar direct connection solution: ‌Hardware interface design‌: The PCIe interface is used to directly connect to the millimeter-wave radar to achieve low-latency transmission of raw data.

[0068] Configure DMA ring buffer (256MB capacity), support duplex transmission mode, and use double buffer mechanism to achieve zero copy transmission. The data packet contains: data packet sequence number, ADC sampling data, check code, attributes, etc.

[0069] Configure TLP (Transaction Layer Packet) priority queues and set three levels of QoS policy: including radar raw data, control instructions, and log transmission.

[0070] Power Management: Independent power supply design: The radar module uses a separate power chip for independent output and is isolated from the main control board.

[0071] Dynamic power consumption adjustment: Switch the supply voltage (3.3V / 5V) according to the working mode to reduce standby power consumption.

[0072] (2) Memory bandwidth optimization of lightweight accelerator for CNN convolutional neural network Memory channel allocation strategy and dynamic bandwidth allocation based on load prediction: aisle Bandwidth allocation Data stream type CH0 40% Neural network weight loading CH1 30% Feature map transmission CH2 20% Sensor raw data CH3 10% System Reserved Data reuse strategy: The Winograd convolution optimization algorithm is used to reduce the number of memory accesses and realize weight shared cache.

[0073] 2. Multimodal perception fusion algorithm (1) Spatiotemporal joint calibration algorithm ‌Hardware Synchronous Trigger‌: Align the LiDAR 4 and camera clocks based on the PPS signal. Establish a timestamp compensation model: ; Coordinate system transformation matrix optimization: based on Lie group SE(3) optimization, Kalman filter prediction, IMU data fusion optimization of installation deviation and vibration interference.

[0074] (2) Transformer-based obstacle classification model Dynamic Attention Mechanism Design: The dynamic attention mechanism is like teaching the model to "dynamically adjust the magnifying glass" - automatically identifying which features (such as human figures, objects, and potholes) are more important, and dynamically adjusting the weights of obstacles based on the current scene to achieve obstacle avoidance and detour.

[0075] ‌Multimodal feature fusion architecture‌: A[LiDAR feature]-->C{CrossAttention} B[Visual Features]-->C{CrossAttention} C-->D[Fusion Features] D-->E[Dynamic obstacle detection head] D-->F[Static obstacle detection head] D-->G[Traversable region segmentation head].

[0076] 3. Post-reliability test analysis End-to-end latency testing: Verify the perception-decision closed-loop latency using a motion capture system.

[0077] ‌‌Power consumption monitoring‌: Record peak power consumption using a recording tool.

[0078] 7. Communication Protocol Implementation Plan 1. RF system hardware design (1) UWB positioning module antenna array Array topology design: A rectangular planar array is used, and phase center calibration is performed, including approach scanning, error compensation, and real-time correction.

[0079] (2) LoRa module anti-interference ‌Line access filter, stronger anti-interference ability.

[0080] PCB layout: RF trace impedance is controlled at 50Ω, ensuring smoother data transmission.

[0081] Shield cavity design: copper shielding cover with grounding via array.

[0082] 2. Communication protocol stack development (1) MQTT QoS dynamic adjustment strategy ‌Network status monitoring indicators‌: parameter Threshold action RSSI>-70dBm QoS0 Maximum throughput mode -85dBm<RSSI≤-70dBm QoS1 Balanced Mode RSSI≤-85dBm QoS2 High reliability mode Configure the adaptive algorithm to enter QoS 0 when the RSSI is greater than -70dBm and the packet loss is less than 0.05.

[0083] When the RSSI is between -85dBm and -70dBm and the packet loss is less than 0.1, it enters QoS1.

[0084] When the RSSI is lower than -85dBm, it enters QoS2.

[0085] Note: The larger the number, the worse the signal.

[0086] (2) Implementation of breakpoint resume mechanism‌ ‌State snapshot storage structure‌ Data packet: contains the task unique ID, last valid timestamp, pose, current target point partition, checksum, etc.

[0087] ‌Recovery Process‌: Main control system->>Cloud: Send recovery request (including the last valid task ID); Cloud-->>Master control system: Returns the latest snapshot data; Main control system->>navigation system: load pose and map blocks; Navigation system->>Re-plan route.

[0088] 3. Post-reliability test analysis (1) UWB positioning accuracy test Multipath environment testing; (2) LoRa anti-interference test ‌Co-channel interference test configuration‌; (3) Protocol stack stress test Network switching simulation tests the QoS switching success rate.

[0089] 8. Safety Control System Development Plan ‌1. Redundant braking system hardware development‌ (1) Mechanical brake control Adopt MCU chip for independent control; (2) Design of acousto-optic devices for the four-level early warning system Optimization of strobe LED driver circuit: Using constant current drive solution to achieve 0-1000Hz adjustable strobe; Multi-level alarm strategy: Warning level Trigger Conditions Response method Level 1 (Attention) Humanoid objects >5m Single beep Level 2 (Warning) Humanoid object 2-5m High-frequency beep + yellow light strobe Level 3 (Emergency) Humanoid objects <2m Pulsed sound and light Level 4 (Collision) Humanoid objects <0.5m emergency braking 2. Security control software development (1) Obstacle avoidance logic based on formal verification The TLA+ modeling framework verifies the completeness of state transitions through the TLC model checker, completes switching between different warning levels, and uses different obstacle avoidance strategies.

[0090] ‌Safe State Machine Design‌: A[Initialization]-->B[Environmental Perception] B-->C {hazard level determination} C-->|Level 1-2|D[Warning Tips] C-->|Level 3|E[Active Avoidance] C-->|Level 4|F[Emergency Brake] (2) Real-time guarantee strategy ‌Interrupt priority configuration‌: Interrupt Source Priority Maximum response time collision sensor 0 (highest) <2μs Braking control 1 <5μs Environmental Perception 2 <10μs System Monitoring 3 <50μs Task scheduling optimization: Adopting a hybrid scheduling strategy (RM+EDF): Key tasks: fixed priority scheduling; Non-critical tasks: Earliest deadline first.

[0091] 3. Post-reliability test analysis (1) SIL (Software in the Loop) testing: Building fault injection test scenarios.

[0092] (2) HIL (Hardware-in-the-Loop) verification: Emergency braking response time <150ms.

[0093] 9. Collaborative Control System Development Plan 1. Multi-device networking hardware development (distributed collaborative architecture) (1) UWB positioning system Base station cluster deployment plan: Adaptive topology: Adopting dynamic self-organizing network technology, the system automatically switches the main base station according to the real-time location of the mobile truck, ensuring a positioning network coverage rate of ≥99%.

[0094] Anti-multipath interference design: Hardware level: Each base station is equipped with an antenna array to suppress multipath effects through beamforming.

[0095] Software level: Adopt CIR (channel impulse response) fingerprint library matching algorithm to eliminate NLOS errors.

[0096] (2) FPGA task scheduling accelerator design Hardware architecture innovation: Parallel pipeline structure: Task scheduling is decomposed into three stages: resource checking (32 threads in parallel), priority sorting (radix sort hardware acceleration), and conflict detection (implemented by Bloom filter).

[0097] Dynamic reconfiguration mechanism‌: FPGA is dynamically reconfigured to achieve performance improvement.

[0098] 2. Swarm intelligence algorithm development (multi-machine collaborative optimization) (1) Auction Mechanism Task Allocation Model ‌3D bidding space modeling‌: ; Dynamic weight adjustment: Automatically adjust the weight coefficient (w1~w3) according to the system load. When the load is >70%, the distance cost is given priority.

[0099] (2) Dynamic Priority Negotiation Protocol Conflict resolution state machine: A[Conflict Detection]-->B[Conflict Type] B-->|Resource Contention|C[Combinatorial Auction] B-->|Path conflict|D[Time and space window negotiation] C-->E[Winner Pays Mechanism] D-->F[Speed ​​Profile Adjustment] Positioning enhancement technology UWB+IMU fusion positioning: Technical elements Implementation Coarse positioning UWB base station group TDoA measurement (update rate 100Hz) Precise positioning IMU data pre-integration compensation (error < 0.1° / h) Calibration mechanism Every 30 seconds, the lidar point cloud is matched to the global map 3. Post-reliability test analysis (1) Multi-machine collaborative handling test (2) Extreme environment testing Warehouse environment testing: Dynamic obstacle density: 15 / 100㎡; Communication interference sources: 3 2.4GHz Wi-Fi hotspots; Test system performance: positioning error, task allocation success rate, and emergency braking response time.

[0100] 10. Optimization and Upgrade Plan 1. Hardware modular expansion (1) Quick-release mechanical interface design Interface structure design: Locking mechanism: Locking quick connection solution.

[0101] ‌Dust-proof power contact design‌: Electrical contacts: IP67 protection (gold-plated contacts + silicone sealing ring); Signal transmission: Supports USB3.0+CAN bus dual-channel redundancy compatibility verification.

[0102] Extendable fork assembly, lifting platform, grabbing fixture.

[0103] (2) Online upgrade system implementation Dual BankFlash architecture‌: Storage partitioning strategy‌: BankA: runs the firmware (256MB, with ECC checksum); BankB: Upgrade image (256MB, supports differential updates).

[0104] OTA upgrade process‌: A[Firmware package signature verification]-->B[Integrity check] B-->|through|C[write to BankB] B-->|Failed|D[Retransmit Request] C-->E[Restart Switch Bank] 2. Software Continuous Learning Mechanism Incremental SLAM algorithm upgrade‌ Dynamic map update mechanism: ‌Hierarchical storage structure‌: Basic layer: static obstacles (storage accuracy ±2cm); Dynamic layer: temporary objects (update frequency 10Hz); Semantic layer: marking areas (loading and unloading areas / no-go areas).

[0105] ‌Optimized feedback mechanism‌: Anomaly Detection‌: Using LSTM networks to predict equipment health, it can provide 200 hours of advance warning of faults such as bearing wear.

[0106] Parameter Tuning: Dynamically adjust PID control parameters based on reinforcement learning.

[0107] 3. Post-reliability test analysis (1) Modular extension testing Module stability test. (2) System upgrade reliability Power Failure Test: This test simulates 10 random power failures during the upgrade process and measures the system recovery success rate.

[0108] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the foregoing embodiments. The foregoing embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. An adaptive terrain-shifting vehicle based on multimodal AI navigation, comprising a tracked chassis drive mechanism, a multimodal sensing module, a power and energy module, a human-machine interaction module, a handling actuator, and a map generation unit, is characterized by: The crawler-type chassis driving mechanism comprises a chassis body (1), the outer side wall of the chassis body (1) is provided with a lock buckle (2), crawler bodies (5) are provided on both sides of the chassis body (1), a plurality of gears (6) are provided inside the crawler body (5), a motor body (8) is installed inside the chassis body (1), the chassis body (1) has a built-in reinforcement frame, and the front and rear ends of the chassis body (1) are provided with standardized mechanical interfaces; six groups of embedded pressure sensor arrays are respectively installed on the crawler bodies (5) on both sides of the chassis body (1) for real-time monitoring of ground pressure distribution; The multimodal sensing module includes an infrared sensor (3), an infrared receiver (31), an ultrasonic sensor, a binocular camera, a laser radar (4), and a cliff sensor (92). The rear end and both sides of the chassis body (1) are equipped with infrared sensors (3). The infrared receiver (31) is integrated into the front end of the vehicle body for receiving infrared signals and ground infrared point tracking. The ultrasonic sensor has a detection range of 0.2m-5m. The binocular camera is used for AI image recognition of obstacles and potholes. The top of the chassis body (1) is equipped with a laser radar (4), and the laser radar (4) builds an environmental map based on the SLAM algorithm. A scanner (9) is provided at the middle position of the bottom of the chassis body (1). Cliff sensors (92) are provided at both ends of the scanner (9), and speakers (91) are provided on both sides of one of the cliff sensors (92). The cliff sensor (92) obtains the size of the transported object and the bottom support point data through the laser ranging module. Collision sensors (7) are provided on both sides of the front end of the chassis body (1). The power and energy module includes a dual-motor drive unit and an intelligent power management unit. The two motor bodies (8) independently control the crawler bodies (5) on both sides to achieve differential steering and in-situ rotation. The intelligent power management unit is equipped with a lithium battery pack and a low-power chipset, and supports automatic recharging through RFID tag positioning. The human-computer interaction module includes a voice prompt unit and a mobile phone communication interface. The voice prompt unit triggers a safety warning voice when it detects people around. The mobile phone communication interface is connected to the mobile terminal through the operator's customized IoT card. The transport actuator includes a rotatable tray, the tray is equipped with a pressure sensor to prevent overloading, and the bottom scanning data is used to calibrate the tray height and angle in real time; the rotatable tray includes a bracket (10) and a universal ball tray (101), manual locks are provided on both sides of the bracket (10) for installation on the chassis body (1), and the top of the bracket (10) is provided with a universal ball tray (101); The map generation unit, based on the SLAM algorithm, integrates inertial navigation and lidar (4) data to generate high-precision two-dimensional terrain maps.

2. The adaptive terrain-shifting vehicle based on multimodal AI navigation according to claim 1 is characterized by: The chassis adopts an expandable crawler chassis. When carrying heavy objects, the system automatically adjusts the track tension and motor torque distribution according to the pressure sensor array data.

3. The multimodal AI navigation-based adaptive terrain-shifting vehicle according to claim 2 is characterized by: The multimodal sensing module also includes a 360-degree surround-view binocular camera group, a millimeter-wave radar, a ToF laser array, and a structured light scanning module. The 360-degree surround-view binocular camera group consists of four groups of 200-degree wide-angle cameras, which combine the millimeter-wave radar and the ToF laser array to build a three-dimensional environmental model; the structured light scanning module uses 100Hz high-frequency pulses to scan the bottom contour of the transported object to generate three-dimensional point cloud data.

4. The adaptive terrain-adaptive heavy vehicle collaborative control system based on multimodal AI navigation is characterized by: It is used for the collaborative control of adaptive terrain-shifting heavy vehicles based on multimodal AI navigation as described in any one of claims 1-3, and the collaborative control system includes: The motion control logic unit has an infrared following mode and a remote control following mode. The infrared following mode analyzes the light spot coordinates through an infrared receiver (31) and dynamically adjusts the crawler speed using a PID algorithm to maintain a relative distance error of ≤5cm from the target point. The remote control following mode is based on UWB positioning technology, calculates the azimuth and distance between the remote control and the vehicle body in real time, and plans the following path in combination with an AI obstacle avoidance algorithm. The intelligent navigation system unit includes a map construction and path planning module. The map construction uses a rasterized map to store terrain information, uses the A* shortest path algorithm to generate a global path, and the DWA algorithm to achieve local dynamic obstacle avoidance. When the user clicks the target point through the mobile phone interface, the system automatically calls historical map data to calculate the optimal route. The safety control strategy unit includes a handling calibration module and a human-machine collaborative safety module. The handling calibration module monitors the center of gravity offset of the cargo in real time through a bottom scanner (9) and controls the posture of the pallet through PID to ensure handling stability with an inclination angle of ≤3°. When the infrared thermal imaging detects that a human body enters the range of 1m, the human-machine collaborative safety module controls the load moving vehicle to slow down to 0.2m / s and continuously broadcasts a prompt voice. The data communication architecture unit uses the MQTT communication protocol to achieve data synchronization between the vehicle and the mobile phone, encrypted transmission of map data, and supports offline task caching and breakpoint resumption functions.

5. The multimodal AI navigation-based adaptive terrain-adaptive heavy vehicle collaborative control system according to claim 4 is characterized by: The obstacle avoidance decision module of the motion control logic unit fuses the data of the ultrasonic sensor, binocular camera and laser radar (4), classifies the obstacle type through a convolutional neural network, and adopts a priority avoidance strategy.

6. The adaptive terrain-adaptive heavy vehicle collaborative control system based on multimodal AI navigation according to claim 5 is characterized by: The intelligent navigation system unit also has an automatic recharging mechanism. By pre-storing the charging room coordinates and real-time positioning data, it triggers the return instruction to enable energy-saving mode and turn off non-essential loads.

7. The multimodal AI navigation-based adaptive terrain-adaptive heavy vehicle collaborative control system according to claim 6 is characterized by: The safety control strategy unit also includes a four-level early warning mechanism: when a moving obstacle is detected 3 meters away, the LED warning light flashes; When a static obstacle is detected within 1.5 meters, an audible and visual alarm is activated; In the 0.5m emergency braking zone, the power output is cut off and the electromagnetic brake is activated; When the tilt angle of the mobile truck is greater than 15°, the hydraulic suspension is automatically locked to prevent rollover.

8. The multimodal AI navigation-based adaptive terrain-adaptive heavy vehicle collaborative control system according to claim 7 is characterized by: The system upgrades emergency obstacle avoidance instructions via LoRa transmission, ensuring emergency response within 20ms. The intelligent path planning algorithm introduces a spatiotemporal joint optimization model, based on a real-time map constructed by SLAM, integrating historical terrain data, and dynamically adjusting the algorithm weight coefficient to achieve an optimal balance between path length and safety distance.

Images