A method and system for adaptive traversing unstructured environments using UTV based on human-machine co-driving and teaching-based learning.

Abstract

This invention provides a method and system for adaptive traversing unstructured environments using UTVs based on human-machine co-driving and teaching-based learning. The method includes: real-time acquisition of multimodal data for perception fusion and physical performance boundary identification; upon triggering a human intervention request signal, simultaneously recording the environmental observation sequence and the human expert's control command sequence, and extracting human control preference feature vectors; training a policy correction model based on a residual architecture, coupling the pre-trained basic policy output with the residual correction term to obtain the final control command at the current moment; performing online policy verification based on a shadow mode, updating the verified residual network weights to the onboard computing unit via a wireless transmission protocol, and automatically invoking the corresponding residual module when encountering similar physical textures or dynamic feedback to achieve convergence and performance optimization of control policy parameters. This invention achieves closed-loop evolution of the control policy.

Description

Technical Field

[0001] This invention relates to the field of autonomous navigation and embodied intelligent control technology for mobile robots, and in particular to a method and system for adaptive traversing unstructured environments using UTVs based on human-robot co-driving and teaching learning. Background Technology

[0002] With the increasing demand for field operations, mountain search and rescue, and automated agriculture, UTV autonomous driving technology has become a hot topic in robotics research. However, in typical unstructured environments such as orchards, forests, and deserts, existing technologies have the following significant drawbacks: (1) Severe disconnect between geometric perception and physical property perception: Existing navigation algorithms for unstructured environments mostly rely on LiDAR or visual sensors to construct elevation maps. However, such methods can only identify the geometry of the terrain and cannot perceive its physical and mechanical properties. For example, when facing deep mud, slippery grass, or loose sand, the geometric features may appear flat, but the actual adhesion coefficient is extremely low. Traditional algorithms often lead to severe slippage or getting stuck in seemingly flat areas due to the lack of in-depth modeling of vehicle-ground interaction, failing to achieve true physical robustness.

[0003] (2) Generalization Bottleneck of End-to-End Learning Models in Out-of-Distribution Scenarios: In recent years, deep learning-based end-to-end control models have performed well in simulators or specific scenarios. However, in actual field applications, unstructured environments have infinite variations (such as soil shear strength under different humidity levels, hidden obstacles under vegetation cover). When the system encounters out-of-distribution (OOD) perturbations not covered by the training set, the output of the black-box neural network is often unpredictable and lacks physical constraints. Existing retraining processes usually need to be performed offline, which cannot enable vehicles to absorb new knowledge and correct control deviations in real time at the work site, resulting in low survivability of the system in long-tail scenarios.

[0004] (3) Low utilization rate of human intervention data and the misconception of "takeover equals failure": In the current shared autonomy framework, human intervention is usually regarded as a functional failure of the autonomous driving system. Existing technical solutions often simply record the trajectory after human intervention, or directly replace the algorithm output with the manual operation as a hard switch. This approach ignores the "control intuition" inherent in human experts when dealing with extreme conditions (such as finding tire adhesion through rapid back-and-forth steering, using inertia to climb hills, and other nonlinear operations). Existing technologies lack an effective mathematical mechanism to transform human corrective behavior into quantifiable control residuals, thus failing to achieve a closed loop from "human intervention" to "strategy evolution".

[0005] (4) Mismatch between static programming algorithms and dynamic physical environment evolution: The physical state of unstructured environments changes dynamically over time (e.g., a sudden drop in friction due to rainfall). Traditional model predictive control (MPC) or path planners typically use fixed dynamic parameters. When the physical characteristics of the environment fluctuate drastically, the preset dynamic model will produce severe prediction bias (model mismatch). Existing adaptive algorithms are either too computationally complex to run in real time on embedded devices, or have limited adjustment range, making it difficult to cope with extreme nonlinear dynamic states such as extreme compression of the suspension system or freewheeling of the drive wheels.

[0006] (5) Lack of a takeover warning mechanism based on physical boundaries: Most autonomous driving systems only request takeover when visual perception fails (such as occlusion or sudden changes in lighting), ignoring warnings at the underlying physical execution level. In off-road conditions, vehicles often reach physical performance boundaries (such as motor torque reaching saturation or rollover limits) while visibility is still clear. Existing technologies lack a comprehensive assessment mechanism that combines proprioceptive feedback and extraprioceptive perception, failing to guide human intervention in a timely manner at the initial stage of physical risks, resulting in takeover timing often lagging behind the accident point.

[0007] In summary, developing a UTV adaptive traversal system capable of real-time identification of physical performance boundaries, in-depth mining of the value of artificial teaching residuals, and online self-evolution of control strategies has become a key technological requirement for improving the operational efficiency of unmanned systems in unstructured environments. Summary of the Invention

[0008] In view of this, the present invention provides a method and system for adaptive traversing unstructured environments using UTV based on human-machine co-driving and teaching learning, in order to solve the above problems.

[0009] This invention provides a method for adaptive traversing unstructured environments using UTVs based on human-machine co-driving and teaching-based learning. The method includes: real-time acquisition of multimodal data using multi-source heterogeneous sensors, followed by perception fusion to obtain a state vector of multidimensional physical features; physical performance boundary identification based on the state vector; triggering a human intervention request signal when the function value of the physical performance boundary evaluation function exceeds a preset safety threshold; establishing a high-frequency asynchronous data buffer at the instant the human intervention request signal is triggered, synchronously recording the environmental observation sequence and the human expert's control command sequence, and extracting a human control preference feature vector using an inverse reinforcement learning framework; generating residual labels based on the state vector, the human expert's control command sequence, the human control preference feature vector, and the predicted control commands of the basic strategy under the same state, and training the system using these residual labels. Alternatively, the policy correction model based on the residual architecture can be updated to obtain residual correction terms. The output of the pre-trained basic policy is coupled with the residual correction terms through dynamic gain to obtain the final control command at the current moment. Before the residual model is deployed, online policy verification is performed based on shadow mode. The final control command is compared with the control command of human experts in real time. When the policy consistency evaluation index is continuously higher than the evolution threshold and the residual network weight update amount is lower than the stability threshold, the residual policy is determined to be safe for online deployment. The verified residual network weights are updated to the on-board computing unit through a wireless transmission protocol. The system establishes a long-term memory library and parameterizes and solidifies the residual fragments of successful escape. When the vehicle encounters similar physical textures or dynamic feedback again, the system automatically calls the corresponding residual module to achieve convergence of control policy parameters and performance optimization.

[0010] In another implementation of the present invention, the state vector of the multidimensional physical feature is represented as:

[0011] in, and These represent the vehicle's three-dimensional position and Euler angles in the local navigation system, respectively. and The linear velocity and angular velocity vectors are fused using Kalman filtering; The instantaneous slip ratio of each drive wheel is calculated based on the wheel angular velocity. With the longitudinal speed of the vehicle Nonlinear mapping relationship; For the real-time compression stroke of the suspension system; This refers to the instantaneous output torque of the actuator. This is the equivalent adhesion coefficient of the ground surface estimated online based on the tire-ground interaction model.

[0012] In another implementation of the present invention, the expression for the physical performance boundary evaluation function is:

[0013] in, These are non-negative weighting coefficients; Roughness operators representing local terrain; Represents the set of slip ratios of the driving wheels. Indicate its variance; The number of drive wheels or actuators participating in the evaluation; and These represent the instantaneous output torque and the rated limit torque of the actuator, respectively.

[0014] In another implementation of the present invention, the expression of the final control instruction is:

[0015] in, These are the parameters of the base model that are frozen. For the final control strategy; Basic strategy; For physical performance boundary evaluation function Ψ ( The determined dynamic gain factor; A nonlinear fusion operator representing the control vector; This represents the residual correction term generated based on the current state and human control preferences; To control the preference vector; For the strategy correction model, θ These are the parameters of the residual network to be optimized.

[0016] In another implementation of the present invention, the expression for the optimization objective function of the strategy correction model is:

[0017] in, For the control instructions of the human expert at time k; The basic model for time k; The dynamic gain factor is determined by the physical performance boundary evaluation function at time k. For a moment The vehicle dynamics state vector; For the extracted control preference features; These are the optimal parameters from the previous time step; This is the regularization coefficient.

[0018] In another implementation of the present invention, the sequence of control instructions from the human expert is represented as follows:

[0019] in, For the longitudinal desired velocity, For the steering angular velocity, Distribute the torque offset factor to the four wheels. This refers to the braking pressure.

[0020] In another implementation of the present invention, the expression for the strategy consistency evaluation index is:

[0021] in, A diagonal matrix for controlling the weights; For period t The output command; For period t Control instructions from human experts; T Indicates the total control cycle.

[0022] Another aspect of the present invention provides a UTV unstructured environment adaptive traversal system based on human-machine co-driving and teaching learning, comprising: a data acquisition module: using multi-source heterogeneous sensors to acquire multimodal data in real time and perform perception fusion to obtain a state vector of multidimensional physical features; a boundary recognition module: performing physical performance boundary recognition based on the state vector, and triggering a human intervention request signal when the function value of the physical performance boundary evaluation function exceeds a preset safety threshold; and a human command extraction module: at the moment the human intervention request signal is triggered, the system establishes a high-frequency asynchronous data buffer, synchronously records the environmental observation sequence and the control command sequence of human experts, and extracts the human control preference feature vector through an inverse reinforcement learning framework. Command Correction Module: Based on the state vector, the control command sequence of human experts, the human control preference feature vector, and the predicted control command of the basic policy under the same state, residual labels are generated. The residual labels are used to train or update the policy correction model based on the residual architecture to obtain residual correction terms. The output of the pre-trained basic policy is coupled with the residual correction terms through dynamic gain to obtain the final control command at the current moment. Policy Verification Module: Before the residual model is deployed, online policy verification is performed based on shadow mode. The final control command is compared with the control command of human experts in real time. When the policy consistency evaluation index is continuously higher than the evolution threshold and the residual network weight update amount is lower than the stability threshold, the residual policy is determined to be safe for online deployment. Update Module: The verified residual network weights are updated to the on-board computing unit through wireless transmission protocol. The system establishes a long-term memory library and parameterizes and solidifies the residual fragments of successful escape. When the vehicle encounters similar physical textures or dynamic feedback again, the system automatically calls the corresponding residual module to achieve convergence of control policy parameters and performance optimization.

[0023] In another aspect, the present invention provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, performs the steps of the method as described in any of the preceding claims. In another aspect, the present invention provides a computer storage medium storing a computer program that, when executed by a processor, implements the steps of the method as described in any of the preceding claims.

[0024] The UTV unstructured environment adaptive traversal method based on human-machine co-driving and teaching learning of the present invention has the following beneficial effects: (1) Achieving deep integration of geometric perception and physical mechanical properties, eliminating "invisible physical traps": In response to the problem of vehicles getting stuck due to the disconnect between geometric perception and physical properties mentioned in the background technology, an internal sensing mechanism based on slip ratio, suspension compression and motor torque feedback is introduced to estimate the equivalent adhesion coefficient of the ground surface and soil shear strength in real time. The system can identify physical risks that cannot be detected by vision, significantly improving the physical robustness of all-terrain vehicles in unstructured environments such as orchards and mountains.

[0025] (2) The generalization bottleneck in out-of-distribution (OOD) scenarios is effectively overcome by using a residual learning architecture: In response to the shortcomings of traditional end-to-end models, such as unpredictable output and high retraining costs when encountering unseen long-tail scenarios, a decoupled architecture based on residual learning is adopted. The system does not need to update the large-scale basic model. It only needs to learn the control compensation amount for specific physical failure modes. This incremental learning method greatly reduces the algorithm's dependence on the amount of samples, enabling the vehicle to quickly adapt to various extreme working conditions with extremely low computational overhead.

[0026] (3) It greatly enhances the technical value of human intervention data and realizes a closed loop from "human correction" to "strategy evolution": It changes the inefficient mode of treating human intervention as a system failure and simply recording the trajectory in traditional technology. It uses the spatiotemporal attention mechanism to deeply explore the control intuition of human experts when dealing with conditions such as getting stuck or going uphill, and transforms it into quantifiable mathematical residuals. This makes each human intervention an effective training data for system evolution, realizing the continuous self-iteration of all-terrain vehicle driving capabilities.

[0027] (4) Dynamic compensation for model mismatch caused by physical environment evolution improves the predictability of planning: In response to the problem that static planning algorithms in the background technology cannot adapt to dynamic physical environment, the residual terms of online optimization are injected into the rolling horizon of MPC in real time. By explicitly compensating for displacement deviation caused by gravity lateral slip or soil subsidence in the prediction stage, the control sequence generated by the system has feedforward anti-disturbance characteristics, ensuring that the vehicle can still closely follow the orchard operation path under complex dynamic disturbances.

[0028] (5) A proactive safety early warning mechanism based on physical boundaries was constructed, significantly reducing the risk of hardware damage: Addressing the issue of delayed takeover due to the lack of physical-level early warning in existing technologies, a defined physical performance boundary evaluation function was used to monitor key physical quantities such as motor torque saturation, vehicle roll angle, and slip ratio in real time. The system can trigger a takeover request at the initial stage of physical risk occurrence, and in conjunction with emergency stop monitoring, provides multiple safety guarantees for special vehicles operating in the field.

[0029] (6) The control strategy has been solidified and reused across scenarios through scene fingerprinting technology: Physical texture fingerprinting technology is introduced to associate and store the verified successful escape experience with specific physical environment features. This enables the all-terrain vehicle to directly call the mature residual module in the knowledge base for feedforward control when it encounters similar terrain in the future, and finally realize the convergence of control strategy parameters and performance optimization. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. By reading the detailed description of the embodiments below, the advantages and benefits of the solutions will become clear to those skilled in the art. The accompanying drawings are only for illustrating preferred embodiments and are not intended to limit the present invention. In the accompanying drawings: Figure 1 This is a schematic diagram of a UTV unstructured environment adaptive traversal method based on human-machine co-driving and teaching learning, according to an embodiment of the present invention.

[0031] Figure 2 This is a schematic diagram of the complete system execution flow according to an embodiment of the present invention.

[0032] Figure 3 This is a schematic diagram illustrating the process of generating a takeover request signal and scene fingerprint from multi-source heterogeneous sensor data according to an embodiment of the present invention.

[0033] Figure 4 This is a schematic diagram showing the fusion architecture of the basic strategy and the residual correction network, and the comparison of the vehicle's off-road trajectory before and after strategy evolution, according to an embodiment of the present invention.

[0034] Figure 5 This is a schematic diagram of a scenario fingerprint matching and strategy reuse process according to an embodiment of the present invention.

[0035] Figure 6 This is a pseudocode diagram of an algorithm based on residual strategy evolution according to an embodiment of the present invention.

[0036] Figure 7This is a schematic diagram illustrating the self-evolutionary control performance of an embodiment of the present invention. Detailed Implementation

[0037] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and thoroughly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art should fall within the protection scope of the present invention.

[0038] Figure 1 This invention provides a schematic flowchart of a method for adaptive traversing unstructured environments using UTV based on human-machine co-driving and teaching-based learning, as shown in the embodiment of the invention. Figure 1 As shown, this embodiment mainly includes: S1. Real-time acquisition of multimodal data using multi-source heterogeneous sensors, followed by perception fusion, yields a state vector of multidimensional physical characteristics.

[0039] S2. Based on the state vector, identify the physical performance boundary. When the function value of the physical performance boundary evaluation function exceeds the preset safety threshold, trigger a manual takeover request signal.

[0040] S3. At the instant the human takeover request signal is triggered, the system establishes a high-frequency asynchronous data buffer, synchronously records the environmental observation sequence and the human expert's control command sequence, and extracts the human control preference feature vector through an inverse reinforcement learning framework.

[0041] S4. Based on the state vector, the control instruction sequence of the human expert, the human control preference feature vector, and the predicted control instructions of the basic policy under the same state, generate residual labels, and use the residual labels to train or update the policy correction model based on the residual architecture to obtain residual correction terms.

[0042] S5. The output of the pre-trained basic policy is coupled with the residual correction term through dynamic gain to obtain the final control command at the current moment.

[0043] S6. Before deploying the residual model, online policy verification is performed based on the shadow mode. The final control command is compared with the control command of human experts in real time. When the policy consistency evaluation index is continuously higher than the evolution threshold and the residual network weight update amount is lower than the stability threshold, the residual policy is determined to have the safety of going online.

[0044] S7. The verified residual network weights are updated to the on-board computing unit via wireless transmission protocol. The system establishes a long-term memory library and parameterizes and solidifies the residual fragments of successful escape. When the vehicle encounters similar physical textures or dynamic feedback again, the system automatically calls the corresponding residual module to achieve convergence of control strategy parameters and performance optimization.

[0045] This invention addresses the shortcomings of existing technologies, such as the disconnect between perception and physical characteristics in unstructured environments, weak generalization ability in OOD scenarios, low utilization rate of manual teaching data, and lack of physical boundary triggering mechanisms. By constructing a physical boundary recognition function, it transforms manual takeover behavior into high-value control residuals, thereby achieving closed-loop evolution of control strategies.

[0046] In another implementation of the present invention, the state vector of the multidimensional physical feature is represented as:

[0047] in, and These represent the vehicle's three-dimensional position and Euler angles in the local navigation system, respectively. and The linear velocity and angular velocity vectors are fused using Kalman filtering; The instantaneous slip ratio of each drive wheel is calculated based on the wheel angular velocity. With the longitudinal speed of the vehicle Nonlinear mapping relationship; For the real-time compression stroke of the suspension system; This refers to the instantaneous output torque of the actuator. This is the equivalent adhesion coefficient of the ground surface estimated online based on the tire-ground interaction model.

[0048] For example, such as Figure 2 As shown, the data stream from the sensing array is acquired in real time via the vehicle-mounted industrial Ethernet bus. The external sensing sensor array includes a 128-line LiDAR mounted on the top of the vehicle and a forward-looking stereo vision sensor, which constructs a local terrain height map. It also extracts surface texture features; the internal sensor array includes a high-precision six-axis inertial measurement unit (IMU), Hall effect encoders at four wheel hubs, a linear variable differential transformer displacement sensor mounted on the suspension arm, and current commands fed back from the motor controller.

[0049] All sensor data is hardware-level timestamped and aligned using a precise time protocol based on IEEE 1588, ensuring that the synchronization error of multimodal data in the time domain is less than 1ms. The system utilizes an extrinsic parameter calibration matrix. All point cloud data, image features, and dynamic parameters are uniformly mapped to the vehicle's centroid coordinate system, completing the spatial dimension representation alignment.

[0050] The system acquires real-time data from the sensing array via the vehicle-mounted bus. In orchard environments, considering foliage obstruction and dust interference, the system utilizes a top-mounted 128-line LiDAR and a forward-looking stereo camera to perform multi-frame fusion, constructing a local height map that includes vegetation density and terrain undulations. Meanwhile, the internal sensing sensors monitor the underlying physical quantities of the UTV in real time, including the angular velocity fed back by the wheel-end Hall encoders and the instantaneous roll and pitch angles fed back by the IMU. All data are aligned at the microsecond level via the PTP protocol, and a Kalman filter algorithm is used to remove high-frequency vibration noise caused by causal road bumps, ensuring the authenticity of the dynamic state vector.

[0051] The system constructs a state vector containing 25-dimensional physical characteristics in real time during each control cycle. The system can perceive the mechanical coupling state of vehicles in unstructured terrain in real time, and calculate the slip ratio of the drive wheels in real time for common terrains such as slippery grass and soft topsoil in orchards. The calculation formula is:

[0052] In actual operation, if the UTV encounters deep mud or loose soil, the system identifies a sudden drop in the surface adhesion coefficient by monitoring the derivative relationship between the motor output current and the slip ratio. When the slip ratio continuously exceeds 0.3 and the longitudinal acceleration approaches zero, the system determines that the vehicle has entered a risk zone of getting stuck, and uses this physical characteristic as an important input for performance boundary identification.

[0053] like Figure 3 As shown, at the instant the trigger signal is generated, the system automatically extracts the external sensory image features and internal sensory dynamic parameters of the current moment, generates a physical texture fingerprint of the specific obstacle scene through a splicing operator, and uses it as an index label for subsequent strategy evolution. At the same time, the system activates the haptic feedback mechanism through the in-vehicle human-machine interface or remote control console, guiding the operator into the teaching mode and realizing a smooth transition of control from the algorithm to human experts.

[0054] In another implementation of the present invention, the expression for the physical performance boundary evaluation function is:

[0055] in, These are non-negative weighting coefficients; Roughness operators representing local terrain; Represents the set of slip ratios of the driving wheels. Indicate its variance; The number of drive wheels or actuators participating in the evaluation; and These represent the instantaneous output torque and the rated limit torque of the actuator, respectively.

[0056] For example, a physical performance boundary evaluation function is defined. This function quantifies the deviation between the current terrain complexity and the vehicle's physical limits in real time. It is a weighted sum of geometric risk, dynamic runaway, rollover stability, and actuator saturation terms, and its mathematical expression is:

[0057] in, These are non-negative weighting coefficients. i Number the drive wheel or actuator. Geometric roughness characterizing local terrain; This is the optimal slip ratio reference value under the current adhesion coefficient; and For real-time roll and pitch angles, and This represents the static rollover limit of the vehicle. and These represent the actual output torque and rated peak torque of each motor. Each weighting coefficient is calibrated through offline dynamic simulation to ensure that the evaluation function accurately reflects the physical failure risk of the vehicle under extreme conditions such as mud and steep slopes.

[0058] The system employs dual-threshold hysteresis comparison logic to evaluate the function. Perform real-time monitoring, when At the safety threshold In the following situations, the system maintains the basic autonomous driving strategy, once... Exceeding the preset performance boundary threshold The system determines that the physical complexity of the current terrain has exceeded the processing boundary of the basic algorithm, and immediately triggers a takeover request signal, actively sending a takeover request to the operator.

[0059] In another implementation of the present invention, the sequence of control instructions from the human expert is represented as follows:

[0060] in, For the longitudinal desired velocity, For the steering angular velocity, Distribute the torque offset factor to the four wheels. This refers to the braking pressure.

[0061] For example, the system performs high-fidelity feature capture of manual intervention control strategies. During the manual takeover phase, the system establishes a high-frequency asynchronous data buffer to synchronously record environmental observation sequences. With human experts' control command sequence The system uses an inverse reinforcement learning framework to map human actions into control preference vectors in the latent feature space. It extracts the control intuition humans have when dealing with situations such as getting stuck or skidding.

[0062] At the instant the trigger signal is generated, the system automatically extracts the external sensory image features and internal sensory dynamic parameters of the current moment, generates a physical texture fingerprint of the specific obstacle scene through a splicing operator, and uses it as an index label for subsequent strategy evolution. At the same time, the system activates the haptic feedback mechanism through the in-vehicle human-machine interface or remote control console, guiding the operator into teaching mode and achieving a smooth transition of control from algorithm to human expert.

[0063] The system maintains a high-frequency asynchronous buffer based on a double-ended queue structure in the background, continuously pushing environmental observation sequences at a frequency of 100Hz. Regarding the system's internal state, once a takeover request signal is triggered and an operator formally intervenes, the system automatically executes a historical backtracking mechanism to capture the state prior to the takeover moment. Seconds (usually set to 3 to 5 seconds) until the end of the takeover. The complete temporal sequence segment. This mechanism ensures that the teaching data includes the results of human operations and the environmental characteristics that caused the system failure, providing complete contextual information for the subsequent residual network.

[0064] Human expert control command sequence Multidimensional extraction and normalization. During the takeover period, the system captures physical control commands issued by the operator in real time via the vehicle bus (CAN-Bus).

[0065] The system adopts a mapping algorithm based on min-max normalization to map the original physical commands to a dimensionless control space of [-1,1], and uses a fifth-order Butterworth low-pass filter to filter out high-frequency jitter noise in manual operation, and extracts a smooth control envelope that conforms to vehicle dynamics constraints.

[0066] Once the operator intervenes, the system automatically rewinds and records the sensor sequence from the five seconds prior to takeover to identify triggering factors that could lead to physical failure. During the takeover, the system records non-standard operator actions at a frequency of 100Hz, such as rapidly turning the wheel left and right (S-shaped maneuvering) in muddy terrain to find lateral traction, or using inertia to overcome tree root obstacles by momentarily increasing throttle. These actions are recorded as a high-dimensional control sequence. .

[0067] Build a policy encoder based on the Transformer architecture. This involves coupling and mapping high-dimensional environmental observation sequences with corresponding control commands to a latent feature space. The encoding process follows a mathematical expression:

[0068] in, The extracted feature vector of human control preferences; For policy encoder The parameters to be learned k The time window length, and These represent the environmental observation sequence and the human expert control command sequence within the time window, respectively. The encoder automatically identifies the strong correlation between environmental features (such as mud depth and slope changes) and control actions (such as instantaneous throttle increase and rapid counter-steering) through a self-attention mechanism. Latent vector It characterizes the control logic of humans when facing physical boundaries, providing high-order feature guidance for residual compensation. The system calculates the actual human commands. With the basic model prediction instructions The difference between them is used to generate residual labels:

[0069] In the orchard stuck-in-the-orchard scenario, the residual label directly quantifies the deficiencies of the basic algorithm in torque distribution or steering compensation. Subsequently, the system stores the encapsulated teaching sample pairs in the experience playback pool, transforming each manual escape process into corrective data that the algorithm can learn from.

[0070] After the takeover is completed, the system will encapsulate the captured features in a structured manner to construct teaching sample pairs. The system generates residual labels. It is defined as the difference between the actual human operating command and the predicted output command of the basic autonomous driving model under the same conditions, that is:

[0071] The residual label directly quantifies the performance loss of the base model under extreme conditions. The system then stores the sample pair in the experience replay pool and classifies and indexes it according to the fingerprint, ensuring that subsequent residual learning can accurately enhance the specific physical failure mode.

[0072] In another implementation of the present invention, the expression of the final control instruction is:

[0073] in, These are the parameters of the base model that are frozen. For the final control strategy; Basic strategy; For physical performance boundary evaluation function Ψ ( The determined dynamic gain factor; A nonlinear fusion operator representing the control vector; This represents the residual correction term generated based on the current state and human control preferences; To control the preference vector; For the strategy correction model, θ These are the parameters of the residual network to be optimized.

[0074] For example, such as Figure 6 As shown, the system performs policy fusion and correction based on residual neural networks, and constructs a policy correction model based on a residual architecture. This architecture ensures the stability of the basic model under normal operating conditions, and compensates for physical model mismatch through residual terms under extreme operating conditions.

[0075] In another implementation of the present invention, the expression for the optimization objective function of the strategy correction model is:

[0076] in, For the control instructions of the human expert at time k; The basic model for time k; The dynamic gain factor is determined by the physical performance boundary evaluation function at time k. For a moment The vehicle dynamics state vector; For the extracted control preference features; These are the optimal parameters from the previous time step; This is the regularization coefficient.

[0077] For example, during UTV operation, the system maintains a fixed-length array in real time. Historical observation sliding window This window is used to store the system state and action pairs within the most recent field of view, providing temporal context information for residual learning. The sliding window... Defined as:

[0078] in, For a moment The vehicle dynamics state vector, Control commands actually executed by human experts. The system extracts control preference features. It uses a rolling view mechanism to focus on physical failure modes under the current terrain features, ensuring local real-time performance of residual compensation.

[0079] Based on the data within the sliding window, construct a residual network with parameters This is a nonlinear least squares optimization problem with decision variables. The objective function is defined as follows: The function consists of a weighted sum of a policy fitting error term and a parameter regularization term:

[0080] The strategy fitting error term measures the current residual parameter. Below, the degree to which the fused control output matches the actions taught by human experts. This forces the network... Absorption Basic Model Unexplained physical deviations. The parameter regularization term constrains the magnitude of parameter updates, where... These are the optimal parameters from the previous time step. This is the regularization coefficient.

[0081] The system introduces a gating factor. To control the intervention intensity of the residual term, The value is determined by the risk assessment value. Decision. Level the road surface in the orchard. Extremely low, The system maintains its basic strategy; when the UTV climbs a steep mountain slope causing reduced front wheel traction... surge, As the residual term rapidly increases, the large transmission ratio steering compensation provided by the residual term is activated, enabling the control strategy to smoothly adapt from normal driving to extreme disturbance rejection.

[0082] Given residual networks It is a deep neural network composed of differentiable operators. The system uses a gradient-based online optimization method to optimize the objective function. Real-time solution is performed. An automatic differentiation engine is used to construct a computational graph and perform backpropagation to calculate the gradient vector of the objective function with respect to the residual parameters.

[0083] The system uses a first-order gradient descent algorithm (such as SGD or AdamW) to update the parameters: Repeat the iteration until the preset maximum number of iterations is reached to output the optimal residual parameter.

[0084] After solving, the system dynamically adjusts the residual terms and weighting factors based on the real-time physical performance boundary evaluation values ​​using a nonlinear mapping operator. This enables a smooth transition from extreme escape mode to a basic stable mode. The system executes a sliding window based on the first-in, first-out (FIFO) principle. The time-domain update pushes in new data and removes the oldest data. For example... Figure 5 As shown, the system extracts the physical texture fingerprint of the current scene. The optimal residual weights are associated with the fingerprint and stored in the vehicle's long-term memory to construct a "scenario-policy" mapping, such as... Figure 4 As shown, this enables the all-terrain vehicle to automatically call the pre-stored residual module when it encounters similar physical feedback in the future, thereby achieving convergence of control strategy parameters and performance optimization.

[0085] A residual-enhanced prediction model coupled with environmental physical characteristics is constructed. The system builds an augmented dynamic model including a non-parametric residual correction term to predict the horizon. Internal extrapolation of vehicle state evolution. In any prediction step... The corrected state deduction equation is defined as follows:

[0086] in, Based on the fundamental kinematic model, the nominal response of the UTV on a hard surface is characterized; residual terms... Explicit compensation was provided for physical mismatches in unstructured scenarios. In the case of slippery grass in an orchard, the residual terms represent the longitudinal slip loss and lateral drift vector. Input features It incorporates the control preferences of human experts when dealing with such working conditions. The predictive model can anticipate the risk of vehicles deviating from the orchard operation path while maintaining nominal commands, providing a physical basis for generating action sequences with feedforward compensation characteristics.

[0087] Construct a comprehensive cost function that couples task objectives, mechanical safety, and control smoothness. Given the action sequence to be optimized Its objective function is defined as:

[0088] in, For physical boundary penalty weights, To control the incremental smoothing weights, To predict the horizon length; state tracking cost ensures the vehicle follows the orchard plant protection path. The physical boundary penalty term introduces real-time evaluation results for hard constraint obstacle avoidance against extreme risks; the control energy term constrains the physical limits of the actuator to prevent hardware thermal failure of the motor due to instantaneous torque exceeding the saturation threshold. This is achieved through a multi-weight matrix. and Through dynamic adjustment, the system achieves an optimal balance between task performance and mechanical lifespan.

[0089] The optimal control sequence is searched in a non-convex space using a sampling-based cross-entropy method. The system initializes a multivariate Gaussian distribution in the action space and samples from this distribution in parallel. The system generates candidate action sequences containing longitudinal velocity and steering angle commands, with the sampling process strictly adhering to the steering physics constraints of the UTV. Utilizing parallel processing capabilities, these sequences are fed into a residual enhancement model for forward inference. A cost function score is used to select the top 10% of samples with the lowest cost value as elite sequences. Based on the statistical characteristics of these elite samples, the system updates the mean and covariance using maximum likelihood estimation, extracting the first action command from the final mean sequence for execution.

[0090] In another implementation of the present invention, the expression for the strategy consistency evaluation index is:

[0091] in, A diagonal matrix for controlling the weights; For period t The output command; For period t Control instructions from human experts; T Indicates the total control cycle.

[0092] For example, the system performs online policy verification based on shadow mode. Before deploying the residual model, the system initiates shadow mode for closed-loop verification. The enhanced model runs in the background, and its output instructions... With the practical operation of human experts Real-time comparisons are performed, and the residual strategy is deemed safe to go live only when it consistently exceeds the evolution threshold.

[0093] In another implementation of the invention, the system solidifies and sustains long-term self-evolution of the control strategy. Verified residual weights are updated to the onboard computing unit via over-the-air (OTA) communication. The system establishes a long-term memory library to parameterize and solidify residual fragments from successful escape attempts. When the vehicle encounters similar physical textures or dynamic feedback again, the system automatically invokes the corresponding residual module to achieve convergence of control strategy parameters and performance optimization.

[0094] For example, the optimal sequence of actions output from the CEM step. Extract the action vector of the first time step. The system maps commands into electrical signals through the vehicle control unit (VCU): longitudinal commands are converted into the PWM duty cycle of the drive motor to generate instantaneous torque to overcome the slope resistance of the orchard; steering commands are converted into the number of steering motor pulses. After the command is issued, the system maintains the control cycle to drive the UTV to generate displacement.

[0095] At sampling time The system acquires new environmental observation images and dynamic states through a sensor array. The system updates the historical observation sliding window according to the FIFO principle. The system pushes the latest state-action pair to the end and removes outdated data from the front. The system ensures that the online optimization process tracks the time-varying characteristics of the environment based on the physical feedback of the nearest horizon.

[0096] Once the system determines that the UTV has successfully traversed the obstacle area, it extracts the physical texture fingerprint at the moment the scene was triggered. The optimal residual network weights are associated with the fingerprint and stored in the vehicle's long-term memory. The system parameterizes scattered escape experience into reusable knowledge modules.

[0097] When the UTV re-enters the unknown area, the system continues to perform scene fingerprint matching and retrieval. If the Euclidean distance between the current fingerprint and the stored fingerprint is less than a preset threshold... The system automatically retrieves the corresponding residual weights from the knowledge base. It also injects into the control loop. This mechanism enables the convergence and performance optimization of control strategy parameters, significantly reduces the frequency of manual intervention, and enables UTV to have continuous self-evolution and zero-sample transfer capabilities.

[0098] Throughout the iterative control process, the system continuously calculates the current state. With mission objectives The similarity is calculated if the distance is less than a threshold. If the crossing is successful, the system will return to cruise mode. Simultaneously, a high-priority safety monitoring thread will be maintained: if the evaluation value during residual strategy execution continues to rise and approaches the mechanical damage limit, the system will immediately trigger a safety emergency stop and forcibly switch back to manual control mode, ensuring that the self-evolution process is controlled within physical safety constraints.

[0099] Example 1 Based on the implementation scheme of this invention, the self-evolutionary performance of the control strategy exhibits significant advantages: such as... Figure 7 As shown, the frequency of manual intervention decreases significantly with the number of operations (training mileage). After accumulating effective manual teaching residual data in the initial stage (first 10 times), compared with the traditional static model where the intervention rate remains high at 0.9, the self-evolutionary method proposed in this invention can smoothly and quickly converge the frequency of manual intervention to below 0.1, verifying that the system has the ability to capture high-fidelity features and evolve strategies efficiently in unstructured long-tail scenarios outside the distribution.

[0100] Another aspect of the present invention provides a UTV unstructured environment adaptive traversal system based on human-machine co-driving and teaching learning, comprising: Data acquisition module: It uses multi-source heterogeneous sensors to collect multimodal data in real time and performs perception fusion to obtain the state vector of multidimensional physical characteristics.

[0101] Boundary identification module: Based on the state vector, physical performance boundary identification is performed. When the function value of the physical performance boundary evaluation function exceeds the preset safety threshold, a manual takeover request signal is triggered.

[0102] Human command extraction module: At the moment the human takeover request signal is triggered, the system establishes a high-frequency asynchronous data buffer, synchronously records the environmental observation sequence and the human expert's control command sequence, and extracts the human control preference feature vector through an inverse reinforcement learning framework.

[0103] Command correction module: Based on the state vector, the control command sequence of human experts, the human control preference feature vector, and the predicted control command of the basic policy under the same state, generate residual labels, use the residual labels to train or update the policy correction model based on the residual architecture, and obtain residual correction terms; couple the output of the pre-trained basic policy with the residual correction terms through dynamic gain to obtain the final control command at the current time.

[0104] Policy verification module: Before the residual model is deployed, online policy verification is performed based on shadow mode. The final control command is compared with the control command of human experts in real time. When the policy consistency evaluation index is continuously higher than the evolution threshold and the residual network weight update amount is lower than the stability threshold, the residual policy is determined to be safe for online deployment.

[0105] Update module: The verified residual network weights are updated to the on-board computing unit via wireless transmission protocol. The system establishes a long-term memory library and parameterizes and solidifies the residual fragments of successful escape. When the vehicle encounters similar physical textures or dynamic feedback again, the system automatically calls the corresponding residual module to achieve convergence of control strategy parameters and performance optimization.

[0106] In another aspect of the present invention, the electronic device includes: a processor, a memory, and a communication bus and a communication interface.

[0107] in: The processor, memory, and communication interface communicate with each other via a communication bus.

[0108] A communication interface is used to communicate with other electronic devices or servers.

[0109] The processor is used to execute programs, specifically the steps of any of the methods described in the above embodiments.

[0110] Specifically, the program may include program code, which includes computer operation instructions.

[0111] The processor may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.

[0112] Memory is used to store programs. Memory may include high-speed RAM, and may also include non-volatile memory, such as at least one disk drive.

[0113] Specifically, the program can be used to cause the processor to execute the steps of any of the methods described in the embodiments. The specific implementation of each step in the program can be found in the corresponding descriptions of the steps and units executed by any of the methods described above, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments.

[0114] An exemplary embodiment of this application also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the methods of various embodiments of this application.

[0115] The methods described above according to embodiments of the present invention can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code originally stored on a remote recording medium or a non-transitory machine-readable medium and subsequently stored on a local recording medium, downloaded via a network. Thus, the methods described herein can be processed by software stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0116] Specific embodiments of the present invention have now been described. Other embodiments are within the scope of the appended claims. In some cases, the actions described in the claims can be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result.

[0117] It should be noted that all directional indications (such as up, down, left, right, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship between the components in a certain order (as shown in the figure). If the specific order changes, the directional indication will also change accordingly.

[0118] In the description of this invention, the terms "first" and "second" are used only for convenience in describing different components or names, and should not be construed as indicating or implying a sequential relationship, relative importance, or implicitly specifying the number of technical features indicated. Thus, a feature defined with "first" and "second" may explicitly or implicitly include at least one of that feature.

[0119] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0120] It should be noted that although specific embodiments of the present invention have been described in detail with reference to the accompanying drawings, this should not be construed as limiting the scope of protection of the present invention. Various modifications and variations that can be made by those skilled in the art without inventive effort within the scope described in the claims still fall within the scope of protection of the present invention.

[0121] The examples of the embodiments of the present invention are intended to concisely illustrate the technical features of the embodiments of the present invention, so that those skilled in the art can intuitively understand the technical features of the embodiments of the present invention, and are not intended to be an improper limitation of the embodiments of the present invention.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for adaptive traversing unstructured environments using UTV based on human-machine co-driving and teaching-based learning, characterized in that, include: Multimodal data is acquired in real time using multi-source heterogeneous sensors and then fused to obtain a state vector of multidimensional physical features. Based on the state vector, physical performance boundaries are identified. When the function value of the physical performance boundary evaluation function exceeds a preset safety threshold, a manual takeover request signal is triggered. At the instant the human takeover request signal is triggered, the system establishes a high-frequency asynchronous data buffer, synchronously records the environmental observation sequence and the human expert's control command sequence, and extracts the human control preference feature vector through an inverse reinforcement learning framework; Based on the state vector, the human expert's control instruction sequence, the human control preference feature vector, and the predicted control instructions of the basic policy under the same state, residual labels are generated. The residual labels are used to train or update the policy correction model based on the residual architecture to obtain the residual correction term. The pre-trained basic policy output is coupled with the residual correction term through dynamic gain to obtain the final control command at the current moment; Before deploying the residual model, online policy verification is performed based on the shadow mode. The final control command is compared with the control command of human experts in real time. When the policy consistency evaluation index is continuously higher than the evolution threshold and the residual network weight update amount is lower than the stability threshold, the residual policy is determined to have the safety of going online. The system updates the verified residual network weights to the onboard computing unit via a wireless transmission protocol. It establishes a long-term memory library and parameterizes and solidifies the residual fragments of successful escape. When the vehicle encounters similar physical textures or dynamic feedback again, the system automatically calls the corresponding residual module to achieve convergence of control strategy parameters and performance optimization.

2. The method according to claim 1, characterized in that, The state vector of the multidimensional physical feature is represented as follows: in, and These represent the vehicle's three-dimensional position and Euler angles in the local navigation system, respectively. and The linear velocity and angular velocity vectors are fused using Kalman filtering; The instantaneous slip ratio of each drive wheel is calculated based on the wheel angular velocity. With the longitudinal speed of the vehicle Nonlinear mapping relationship; For the real-time compression stroke of the suspension system; This refers to the instantaneous output torque of the actuator. This is the equivalent adhesion coefficient of the ground surface estimated online based on the tire-ground interaction model.

3. The method according to claim 2, characterized in that, The expression for the physical performance boundary evaluation function is as follows: in, These are non-negative weighting coefficients; Roughness operators representing local terrain; Represents the set of slip ratios of the driving wheels. Indicate its variance; The number of drive wheels or actuators participating in the evaluation; and These represent the instantaneous output torque and the rated limit torque of the actuator, respectively.

4. The method according to claim 3, characterized in that, The expression for the final control command is: in, These are the parameters of the base model that are frozen. For the final control strategy; Basic strategy; For physical performance boundary evaluation function Ψ ( The determined dynamic gain factor; A nonlinear fusion operator representing the control vector; This represents the residual correction term generated based on the current state and human control preferences; To control the preference vector; For the strategy correction model, θ These are the parameters of the residual network to be optimized.

5. The method according to claim 4, characterized in that, The expression for the optimization objective function of the strategy correction model is: in, For the control instructions of the human expert at time k; The basic model for time k; The dynamic gain factor is determined by the physical performance boundary evaluation function at time k. For a moment The vehicle dynamics state vector; For the extracted control preference features; These are the optimal parameters from the previous time step; is the regularization coefficient.

6. The method according to claim 5, characterized in that, The sequence of control instructions from the human expert is represented as follows: in, For the longitudinal desired velocity, For the steering angular velocity, Distribute the torque offset factor to the four wheels. This refers to the braking pressure.

7. The method according to claim 1, characterized in that, The expression for the strategy consistency evaluation index is as follows: in, A diagonal matrix for controlling the weights; For period t The output command; For period t Control instructions from human experts; T Indicates the total control cycle.

8. A UTV unstructured environment adaptive traversal system based on human-machine co-driving and teaching-based learning, characterized in that, include: Data acquisition module: Real-time acquisition of multimodal data using multi-source heterogeneous sensors, and sensor fusion to obtain state vectors of multidimensional physical features; Boundary recognition module: Based on the state vector, physical performance boundary recognition is performed. When the function value of the physical performance boundary evaluation function exceeds the preset safety threshold, a manual takeover request signal is triggered. Human command extraction module: At the moment the human takeover request signal is triggered, the system establishes a high-frequency asynchronous data buffer, synchronously records the environmental observation sequence and the human expert's control command sequence, and extracts the human control preference feature vector through an inverse reinforcement learning framework; Command correction module: Based on the state vector, the control command sequence of human experts, the human control preference feature vector, and the predicted control command of the basic policy under the same state, generate residual labels, use the residual labels to train or update the policy correction model based on the residual architecture to obtain residual correction terms; couple the output of the pre-trained basic policy with the residual correction terms through dynamic gain to obtain the final control command at the current moment. Policy verification module: Before the residual model is deployed, online policy verification is performed based on shadow mode. The final control command is compared with the control command of human experts in real time. When the policy consistency evaluation index is continuously higher than the evolution threshold and the residual network weight update amount is lower than the stability threshold, the residual policy is determined to be safe for online deployment. Update module: The verified residual network weights are updated to the on-board computing unit via wireless transmission protocol. The system establishes a long-term memory library and parameterizes and solidifies the residual fragments of successful escape. When the vehicle encounters similar physical textures or dynamic feedback again, the system automatically calls the corresponding residual module to achieve convergence of control strategy parameters and performance optimization.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 7.

Images