A smart vehicle trajectory tracking method and device based on visual-tactile fusion and dynamics compensation

Abstract

The application discloses a kind of visual touch fusion and dynamics compensation intelligent vehicle trajectory tracking method and device.The method synchronously collects the road surface visual image data of the pre-foresight area in front of vehicle and the inertial vibration data of vehicle body chassis, respectively extracts road surface texture semantic features and road surface roughness features, and the features are spatiotemporally aligned and fused to obtain real-time road dynamics parameters;Then, according to the real-time road dynamics parameters, the vehicle dynamics prediction model and the dynamics boundary constraint condition are dynamically reconstructed, the optimization control problem is solved, and the vehicle actuator control instruction is output, so as to realize the intelligent vehicle trajectory tracking under complex road conditions.The method can effectively improve the intelligent vehicle trajectory tracking accuracy and environmental adaptability.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving and mobile robot motion control technology, specifically relating to an intelligent vehicle trajectory tracking method and device based on a multimodal sensor fusion architecture with visual-touch fusion and dynamic compensation. Background Technology

[0002] With the rapid development of intelligent transportation and logistics automation technologies, the application scenarios of intelligent vehicles (such as inspection robots and unmanned delivery vehicles) are accelerating their expansion from structured closed environments to complex unstructured outdoor environments. As a core component of the underlying motion control of intelligent vehicles, trajectory tracking's control accuracy and stability directly determine the vehicle's operational efficiency and driving safety. It is of great significance for reducing the risk of loss of control in complex road conditions and ensuring the reliable execution of automated tasks.

[0003] Existing intelligent vehicle trajectory tracking methods mostly rely on single-sensor perception or fixed-parameter vehicle dynamics models, solving for control commands through geometric calculations or model prediction algorithms. However, such algorithms are often based on the assumption of ideal high-adhesion road surfaces, and when faced with unstructured and complex working conditions, they reveal the following significant shortcomings:

[0004] First, the robustness of single-vision perception is insufficient. Pure vision systems are easily deceived by drastic changes in external lighting and complex road textures (for example, misjudging the specular reflection of a puddle as a smooth road surface), causing the feedforward perception model to fail;

[0005] Secondly, relying solely on inertial sensing results in control lag. Tactile / inertial sensing based solely on vehicle body or chassis vibrations often only intervenes passively after the vehicle has already physically skidded or deviated from its trajectory, failing to provide early warning and leading to a severe lag in the underlying control response.

[0006] Third, the dynamic model parameters lack adaptive capability. The physical properties of unstructured pavements (such as the road adhesion coefficient) are highly time-varying. Traditional fixed-parameter dynamic models cannot accurately represent the true nonlinear dynamic state of vehicles on roads with alternating wet and dry conditions, ice and snow, or gravel.

[0007] In summary, the singularity of the sensing mode and the static nature of the dynamic model make existing trajectory tracking algorithms prone to tracking error divergence under complex, time-varying road conditions, potentially even leading to vehicle sideslip and instability. Therefore, there is an urgent need for a trajectory tracking scheme that can achieve complementary advantages from multiple sensing modes and dynamically and adaptively adjust the underlying control parameters. Summary of the Invention

[0008] Purpose of the Invention: Addressing the problems of existing technologies, such as susceptibility to environmental interference with single visual perception, control lag with single inertial feedback, and inability of fixed-parameter dynamic models to adapt to unstructured complex road surfaces, the present invention aims to provide a method and device for intelligent vehicle trajectory tracking that integrates visual-tactile perception and dynamic compensation. This invention overcomes the limitations of single-modal perception by utilizing the cross-modal complementary characteristics of "visual feedforward" and "tactile feedback." Through high-precision spatiotemporal alignment and confidence fusion, it dynamically reconstructs the dynamic constraint boundaries of the underlying control in real time, effectively solving the problems of tracking divergence and sideslip instability in intelligent vehicles on complex road surfaces, and significantly improving the robustness and environmental adaptability of autonomous motion.

[0009] Technical Solution: To achieve the above objectives, this invention provides an intelligent vehicle trajectory tracking method based on visual-touch fusion perception and dynamic compensation, comprising the following steps:

[0010] Step 1: Simultaneously collect visual image data of the road surface in the pre-aiming area in front of the vehicle in the driving environment to be tested, as well as inertial vibration data of the vehicle chassis;

[0011] Step 2: Extract the physical features of each modal data; wherein, a deep learning network is used to extract the road surface texture semantic features of the visual image data, and frequency domain analysis is used to extract the road surface roughness features of the inertial vibration data.

[0012] Step 3: Perform spatiotemporal alignment and confidence fusion of the road surface texture semantic features and the road surface roughness features to obtain real-time road surface dynamic parameters;

[0013] Step 4: Based on the real-time road dynamics parameters, dynamically reconstruct the vehicle dynamics prediction model and dynamic boundary constraints;

[0014] Step 5: Based on the reconstructed vehicle dynamics prediction model and dynamic boundary constraints, solve the optimization control problem and output motor and servo commands to achieve intelligent vehicle trajectory tracking.

[0015] Further, in step 2, the extraction of road surface texture semantic features from visual image data using a deep learning network specifically involves: constructing a lightweight convolutional neural network that incorporates a channel attention mechanism; inputting the visual image data into the lightweight convolutional neural network; and using the convolutional layers to extract multi-channel basic road surface texture feature maps. In lightweight convolutional neural networks, global spatial descriptors are calculated sequentially through the aforementioned channel attention mechanism. With attention weight coefficient For the basic road surface texture feature map Perform feature recalibration and output the recalibrated road surface texture semantic feature map. The processing formula is:

[0016]

[0017]

[0018]

[0019] In the formula, and These represent the height and width of the feature map, respectively. and These are the spatial coordinate indices in the height and width directions, respectively. and These are the weight matrices for the dimensionality reduction and dimensionality increase fully connected layers, respectively. It is the ReLU activation function. It is the Sigmoid activation function. The first one obtained after global average pooling Global space descriptor for each channel, The first time learned from the network Attention weight coefficients for each channel The output after recalibration Semantic feature map of road surface texture for each channel.

[0020] The recalibrated road surface texture semantic feature map is processed using a fully connected layer. The probability distribution of road surface material categories is mapped to the category probability distribution, and the category probability distribution is weighted and calculated as the feedforward road surface reference adhesion coefficient based on the preset physical property lookup table.

[0021] Furthermore, in step 2, the extraction of road surface roughness features from inertial vibration data using frequency domain analysis specifically involves: extracting a Z-axis acceleration data sequence within a preset time window. The Fast Fourier Transform (FFT) algorithm is used to transform it into a form containing the first... Complex amplitude of each frequency component The frequency domain signal is then analyzed; subsequently, based on the preset frequency band division criteria, the power spectral density (PSD) energy value of each frequency band is calculated. To construct the vibration feature vector, the relevant transformation and calculation formulas are as follows:

[0022]

[0023]

[0024] In the formula, The total number of sampling points within the preset time window. For the time-domain sampling point index, and , It is a frequency index, and , The imaginary unit, This represents the division into low-frequency, mid-frequency, or high-frequency bands. and The first The starting frequency index and ending frequency index corresponding to each frequency band are used; the vibration feature vector containing the energy values ​​of the low frequency band, mid frequency band and high frequency band is input into the pre-trained support vector machine (SVM) classifier to identify the real physical roughness of the road surface at the current wheel contact and generate the feedback road surface reference adhesion coefficient.

[0025] Furthermore, step 3 specifically involves: combining the current longitudinal vehicle speed and the front wheel steering state, calculating the actual arc length of the driving trajectory from the aiming point to the current wheel contact surface. And solve for the time delay that satisfies the kinematic integral equation. ,in For integration time variable, Given the front wheel steering angle at the corresponding moment, the kinematic integral equation is as follows:

[0026]

[0027] Based on the time delay Establish a spatiotemporally aligned delay queue and extract from the delay queue. The feedforward road surface reference adhesion coefficient corresponding to the visual semantic features stored at each time step is used as prior information, with the current time step as the basis. The feedback road surface reference adhesion coefficient is used as likelihood information;

[0028] The posterior probability after fusion is calculated using a recursive Bayesian estimation algorithm. The probability update formula is as follows:

[0029]

[0030] In the formula, The real-time pavement dynamic parameters to be estimated are... For the present The observed value at time, To until Historical observation sequence of time, The prior probability is based on historical information. The likelihood probability is based on the current feedback information. The marginal likelihood probability that serves as the normalization factor is selected as the parameter value corresponding to the maximum posterior probability as the final output, and denoted as . .

[0031] Furthermore, step 4 specifically involves: based on real-time road dynamic parameters The front and rear wheel lateral stiffnesses in the two-degree-of-freedom vehicle dynamics differential equations are updated in real time to construct a linear time-varying state space equation:

[0032]

[0033] Wherein, the state vector , State vector The derivative with time controls the input. Front wheel lateral stiffness in state-space equations Rear wheel lateral stiffness All are configured to be about The mapping function, whose mapping update formula is: subscript They represent the front wheel and the rear wheel, respectively. This is the tire reference lateral stiffness proportional coefficient. and These are the system state matrix and control matrix reconstructed based on the updated lateral stiffness, respectively;

[0034] matrix and They are:

[0035] ,

[0036] In the formula, The lateral velocity of the vehicle's center of gravity. The yaw rate is angular velocity. For longitudinal velocity, For the overall vehicle quality, To bypass moment of inertia of the shaft and These are the distances from the center of mass to the front and rear axles, respectively. For the front wheel steering angle, This refers to the vertical load on the corresponding wheel;

[0037] The dynamic friction circle constraint condition based on the road surface adhesion limit is constructed and expressed by the inequality as follows:

[0038]

[0039] In the formula, d represents the front wheel or the rear wheel. For the first The longitudinal force of the wheel, For the first Lateral force of the wheel, For the first Vertical load on the wheel. When As road conditions change, the boundary range of the inequality is automatically adjusted to limit the maximum lateral and longitudinal acceleration of the vehicle.

[0040] Furthermore, step 5 specifically involves: constructing the quadratic programming objective function for Model Predictive Control (MPC). The formula is:

[0041]

[0042] In the formula, Indicates the current Always looking towards the future The predicted step size is the value of the state or input variable. To predict the step size index, This is the predicted state vector for the vehicle. The reference trajectory state vector, The control input increment represents the change in steering angle between adjacent control steps. and These are the weight matrices for the state error and the control increment, respectively. and These are the prediction time domain and the control time domain, respectively. To prevent the search for a relaxation factor that yields no feasible solution, These are the corresponding relaxation factor weighting coefficients;

[0043] Under the premise of satisfying the dynamic friction circle constraint and the physical constraints of the vehicle actuator, the objective function is minimized using a solver. The optimal control increment sequence is obtained, and the first control increment in the sequence is superimposed with the actual control quantity at the previous moment to be used as the motor torque and servo angle command output.

[0044] Furthermore, the method also includes a safety fallback strategy: calculating the conflict entropy value between the road surface texture semantic features and the road surface roughness features; if the conflict entropy value exceeds a preset safety threshold, it is determined that there is a modal conflict in the perception system, and at this time the real-time road surface dynamic parameters are forcibly set to a preset low adhesion coefficient safety value and switched to conservative control mode.

[0045] This invention also discloses an intelligent vehicle trajectory tracking device based on visual-touch fusion and dynamic compensation, comprising:

[0046] The data acquisition module is configured to simultaneously acquire visual image data of the road surface in the pre-aiming area in front of the vehicle in the driving environment to be detected, as well as inertial vibration data of the vehicle chassis;

[0047] The feature extraction module is configured to extract the road texture semantic features of the road visual image data using a deep learning network, and to extract the road roughness features of the inertial vibration data using frequency domain analysis.

[0048] The fusion verification module is configured to perform spatiotemporal alignment and confidence fusion of the road surface texture semantic features and the road surface roughness features to obtain real-time road surface dynamic parameters.

[0049] The control update module is configured to dynamically reconstruct the prediction model and dynamic boundary constraints of the model prediction controller based on the real-time road dynamic parameters.

[0050] The execution output module is configured to solve the optimization control problem based on the reconstructed prediction model and dynamic boundary constraints, and output motor and servo commands to achieve intelligent vehicle trajectory tracking.

[0051] The present invention also discloses a computer device, including a memory, a processor, and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the steps of any of the methods described above.

[0052] The present invention also discloses a computer-readable storage medium having a computer program or instructions stored thereon, characterized in that the computer program or instructions, when executed by a processor, implement the steps of any of the methods described above.

[0053] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0054] First, it breaks through the limitations of single-modal perception and improves the accuracy of all-weather attribute recognition. Compared with pure vision, which is easily deceived, or pure inertial perception, which suffers from serious lag, this invention innovatively constructs a cross-modal complementary mechanism of "visual feedforward + tactile feedback". By introducing a lightweight convolutional network with channel attention (SE) and fusing it with inertial frequency domain features, the estimation error of the road surface physical adhesion coefficient is controlled within ±0.05 under complex optical interference conditions such as ice and snow and water accumulation. The overall perception accuracy is improved by about 15% to 20%, effectively avoiding the failure of the feedforward model caused by single visual "misidentification".

[0055] Secondly, it eliminates the physical spatial misalignment between visual and tactile feedback, achieving "zero-delay" state fusion. Addressing the inherent physical spatiotemporal differences between forward visual anti-sighting and chassis tactile feedback, this invention implicitly introduces a vehicle kinematic curve integral model considering the front wheel steering angle, establishing a spatiotemporally aligned delay queue. Combined with a recursive Bayesian estimation algorithm, this not only completely eliminates the spatial misalignment bias between sensors but also advances the update response time of the underlying dynamic parameters by approximately 150 to 300 milliseconds, reserving ample dynamic compensation time for the underlying control.

[0056] Thirdly, it pioneers adaptive dynamic boundary constraints, significantly suppressing sideslip instability under extreme conditions. This invention breaks through the limitations of using static parameter models in traditional MPC control. Based on high-confidence perception results, it reconstructs the underlying dynamic differential equations in real time and configures the nonlinear friction circle constraint as a dynamically contracting boundary. Simulation and test data show that in extreme off-road conditions where a vehicle suddenly transitions from a high-adhesion dry asphalt road surface to a low-adhesion wet and slippery road surface at a speed of 60 km / h (e.g., the adhesion coefficient abruptly changes from 0.8 to 0.2), this invention reduces the maximum lateral tracking error of the vehicle by more than 30%, and strictly limits the peak value of the centroid sideslip angle to ±3°. 。 Within the safety threshold, it has achieved a fundamental leap from "passive rescue" to "active adaptive avoidance", completely curbing the phenomenon of instability and tail-swing. Attached Figure Description

[0057] Figure 1 This is a flowchart illustrating the overall process of a visual-touch fusion and dynamic compensation method and device for intelligent vehicle trajectory tracking, as provided in an embodiment of the present invention.

[0058] Figure 2 This is a schematic diagram of the lightweight convolutional neural network processing flow that introduces the channel attention mechanism in an embodiment of the present invention;

[0059] Figure 3 This is a schematic diagram of the spatiotemporal alignment and fusion process based on recursive Bayesian estimation in an embodiment of the present invention;

[0060] Figure 4 This is a structural block diagram of an intelligent vehicle trajectory tracking device based on visual-touch fusion perception and dynamic compensation provided in an embodiment of the present invention. Detailed Implementation

[0061] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0062] Unless otherwise specified, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of this disclosure. It should also be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn to actual scale. Techniques, methods, and apparatus known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and apparatus should be considered part of the specification.

[0063] Example 1: A method and device for intelligent vehicle trajectory tracking with visual-touch fusion and dynamic compensation

[0064] To address the problems in existing technologies, such as the susceptibility of single visual perception to environmental interference, the lag in single tactile perception, and the inability of fixed dynamic models to adapt to complex road surface changes, this disclosure proposes an intelligent vehicle trajectory tracking method and device that integrates visual and tactile perception with dynamic compensation.

[0065] like Figure 1 As shown, the method mainly includes the following steps:

[0066] Step S1: Synchronous Acquisition of Multimodal Heterogeneous Data

[0067] Multimodal data is simultaneously acquired for the driving environment to be detected. Specifically, a monocular or binocular camera mounted at the front of the intelligent vehicle with a downward tilt angle is used to acquire visual image data of the road surface in the pre-aiming area in front of the vehicle (the pre-aiming range is set to 5-15 meters ahead, and the sampling rate is configured to 30Hz). Simultaneously, an inertial measurement unit (IMU) mounted on the chassis or suspension is used to synchronously acquire inertial vibration data of the chassis (mainly Z-axis acceleration data; to capture high-frequency road excitation, the sampling rate is configured to 200Hz or even higher). To address the issue of inconsistent sampling frequencies among heterogeneous sensors, this embodiment employs a timestamp-based soft synchronization matching mechanism. Using the low-frequency frames of the camera as a reference, high-frequency IMU data sequences within the corresponding timestamp interval are extracted to ensure alignment of visual and tactile perception on the time scale.

[0068] Step S2: High-dimensional extraction of physical features: Extract the physical features of each modality of data respectively.

[0069] (1) Visual Feature Extraction: The semantic features of road surface texture in visual image data are extracted using a deep learning network. In some embodiments, a lightweight convolutional neural network with a channel attention mechanism (Squeeze-and-Excitation, SE) is constructed. The visual image data is input into the network, and the convolutional layers are used to extract multi-channel basic road surface texture feature maps. .like Figure 2 As shown, in lightweight convolutional neural networks, the global space descriptor is calculated sequentially through a channel attention mechanism. With attention weight coefficient For the basic road surface texture feature map Perform feature recalibration and output the recalibrated road surface texture semantic feature map. The processing formula is:

[0070]

[0071]

[0072]

[0073] In the formula, and These represent the height and width of the feature map, respectively. and These are the spatial coordinate indices in the height and width directions, respectively. and These are the weight matrices for the dimensionality reduction and dimensionality increase fully connected layers, respectively. It is the ReLU activation function. Using the Sigmoid activation function, the network utilizes a Softmax layer to extract the feature maps. The probability distribution is mapped to the road surface material (such as dry asphalt, wet cement, ice and snow, etc.), and based on a pre-defined physical attribute lookup table, the weighted expectation of this probability distribution is calculated to output the feedforward road surface reference adhesion coefficient. This represents the estimated prior value of the road surface that the "eye" sees about to reach.

[0074] (2) Tactile Feature Extraction: Surface roughness features of inertial vibration data are extracted using frequency domain analysis. Z-axis acceleration data sequences within a preset time window (e.g., 100 sampling points corresponding to the past 0.5 seconds) are extracted. The signal was converted from the time domain to the frequency domain using the Fast Fourier Transform (FFT) algorithm. Subsequently, the power spectral density (PSD) energy values ​​for low frequencies (representing road surface undulations), mid frequencies, and high frequencies (representing road surface texture such as gravel) were calculated. To construct a three-dimensional vibration feature vector, the relevant formula is:

[0075]

[0076]

[0077] In the formula, The number of sampling points. For frequency index, For time-domain sampling point index, The imaginary unit, Representing the low-frequency, mid-frequency, or high-frequency bands, the constructed vibration feature vectors are input into a pre-trained Support Vector Machine (SVM) classifier. To handle nonlinearly inseparable data, the SVM in this embodiment uses a radial basis function (RBF) kernel function for high-dimensional mapping.

[0078] ,

[0079] In the formula, For feature vectors, represents the hyperparameters of the kernel function. The SVM is used to identify the true physical roughness of the road surface currently in contact with the wheel, generating a feedback road surface reference adhesion coefficient. This represents the true likelihood value of the current road surface as perceived by the "body".

[0080] It should be noted that, in order to achieve an accurate mapping from high-frequency vertical vibration of the Z-axis to the adhesion characteristics of the horizontal surface, the SVM model in this embodiment is trained based on a pre-collected joint calibration database of "road surface roughness-tire slip adhesion characteristics", thereby establishing a nonlinear mapping relationship from macroscopic physical vibration to peak adhesion coefficient.

[0081] Step S3: Spatiotemporal alignment and confidence Bayesian fusion

[0082] Since visual perception corresponds to road surface information in the area ahead of the vehicle, while inertial vibration feedback corresponds to road surface information at the current wheel contact position, there is a natural misalignment between the two in terms of spatial position and temporal response. Therefore, it is necessary to align and fuse visual and tactile features in time and space.

[0083] First, based on the current longitudinal vehicle speed and front wheel steering status, calculate the actual arc length of the driving trajectory from the aiming point to the current wheel contact surface. And solve for the time delay that satisfies the kinematic integral equation. The kinematic integral equation is as follows:

[0084]

[0085] In the formula For integration time variable, This represents the front wheel steering angle at the corresponding moment;

[0086] Based on the time delay Establish a FIFO space-time aligned delay queue and extract from the delay queue. The feedforward road surface reference adhesion coefficient corresponding to the visual semantic features stored at each time step is used as prior information, with the current time step as the basis. The feedback road surface reference adhesion coefficient is used as observation information. .

[0087] The posterior probability after fusion is calculated using a recursive Bayesian estimation algorithm. The probability update formula is as follows:

[0088]

[0089] In practice, the observed likelihood probability It follows a mean of true state and a variance of sensor measurement noise of . The Gaussian distribution is used, and the parameter values ​​corresponding to the maximum a posteriori probability (MAP) are selected as the final output real-time road dynamic parameters. .

[0090] Step S4: Adaptive Reconstruction of Dynamic Model and Friction Circle Boundary

[0091] Based on the acquired real-time pavement dynamic parameters Dynamically reconstruct the vehicle dynamics prediction model and dynamic boundary constraints.

[0092] First, the front and rear wheel lateral stiffnesses in the two-degree-of-freedom vehicle dynamics differential equations are updated in real time, where the front wheel lateral stiffness... Rear wheel lateral stiffness All are configured to be about The mapping function, whose mapping update formula is: subscript These represent the front and rear wheels, respectively. The vertical load on the wheel is also considered. Based on the overall vehicle weight Calculation of static distribution of front and rear wheelbase:

[0093] Front wheel load Rear wheel load , This is the acceleration due to gravity.

[0094] Constructing linear time-varying state-space equations: , where the state vector , State vector The derivative with time controls the input. , The matrix represents the tire's reference lateral stiffness proportionality coefficient. and They are:

[0095] ,

[0096] Secondly, a dynamic friction circle constraint condition based on the road surface adhesion limit is constructed, which is expressed by the inequality as follows:

[0097]

[0098] This constraint endows the originally static attached ellipse with dynamic elasticity: when When the speed decreases (such as when driving on icy or snowy roads), the boundary range of the inequality will automatically shrink, thereby strictly limiting the maximum lateral and longitudinal acceleration of the vehicle at the physical and mechanical level and actively preventing sideslip.

[0099] Step S5: MPC optimization solution and control command output

[0100] Based on the reconstructed vehicle dynamics prediction model and boundary constraints, the optimization control problem is solved. A quadratic programming objective function for Model Predictive Control (MPC) is constructed. The formula is:

[0101]

[0102] In the formula, Indicates the current Always looking towards the future The predicted step size is the value of the state or input variable to be predicted, and the state deviation vector is used for prediction. Configured to include lateral tracking error With heading angle error The column vector, i.e. The control increment is used to characterize the change in control input between adjacent control moments. To predict the step size index, This is the predicted state vector for the vehicle. The reference trajectory state vector, The control input increment represents the change in steering angle between adjacent control steps. and These are the weight matrices for the state error and the control increment, respectively. and These are the prediction time domain and the control time domain, respectively. To prevent the search for a relaxation factor that yields no feasible solution, These are the corresponding relaxation factor weighting coefficients;

[0103] Under the premise of satisfying the dynamic friction circle constraint and the actuator physical constraint, the objective function is optimized by using a quadratic programming solver to obtain the optimal control increment sequence. The first control increment in the sequence is then used as the vehicle actuator control command output at the current moment to achieve intelligent vehicle trajectory tracking.

[0104] Safety fallback strategy (modal conflict handling):

[0105] To further improve the system's reliability in extreme environments, the method also includes a safety fallback check: calculating the conflict entropy value between the road surface texture semantic features and the road surface roughness features. Specifically, let the road surface category probability distribution vector output by the visual network be... The class probability distribution vector output by the tactile SVM ,in This represents the total number of road surface categories. Define the conflict entropy value between the two. Calculation based on symmetric cross-entropy:

[0106]

[0107] when When the value exceeds the preset safety threshold, it indicates a serious discrepancy between visual feedforward and tactile feedback, and the system determines that there is a modal conflict. In this case, the real-time road dynamics parameters are forcibly changed. Set to a preset low adhesion coefficient safety value (e.g., 0.2) and switch to conservative control mode.

[0108] Example 2: An intelligent vehicle trajectory tracking device based on visual-touch fusion perception and dynamic compensation

[0109] like Figure 4 As shown, this embodiment provides an intelligent vehicle trajectory tracking device based on visual-touch fusion perception and dynamic compensation. This device can be executed by an onboard computing platform (such as an NVIDIA Jetson series or an industrial control computer) and includes:

[0110] 1. Data Acquisition Module: Configured to simultaneously acquire visual image data of the road surface and inertial vibration data of the vehicle chassis. This module typically includes a camera driver interface and an IMU data readout interface.

[0111] 2. Feature Extraction Module: Configured to extract visual semantic features using deep learning networks and tactile vibration features using frequency domain analysis. This module embeds a lightweight convolutional neural network acceleration unit (such as the TensorRT acceleration engine) and a fast Fourier transform calculation unit (such as a DSP or FPGA hard core).

[0112] 3. Fusion and Verification Module: Configured to perform spatiotemporal alignment and confidence verification, fusing and generating real-time road dynamic parameters. This module maintains a circular buffer as a delay queue and runs a recursive Bayesian estimation algorithm.

[0113] 4. Control Update Module: Configured to dynamically update the prediction model matrix and dynamic boundary constraints of the model prediction controller based on real-time road dynamic parameters.

[0114] 5. Output Module: Configured to solve optimization control problems and output motor and servo commands. This module communicates with the underlying drive-by-wire chassis via CAN bus or PWM signal.

[0115] Based on the same technical solution, this disclosure also relates to a computer device. The device includes a processor, memory, communication interface, and input / output interface connected via a system bus.

[0116] Processors: may include high-performance GPUs (for deep learning inference of visual textures) and CPUs (for logic operations for Bayesian fusion and model prediction control).

[0117] Memory: Stores non-volatile program instructions, which, when executed by the processor, implement the trajectory tracking method described above.

[0118] Input / output interfaces: connect to the vehicle-mounted camera, inertial measurement unit (IMU), and underlying motor and servo drive.

[0119] In addition, this disclosure also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0120] The beneficial effects achieved by this invention are as follows: Compared with the prior art, this invention has the following significant technical advantages:

[0121] First, it breaks through the limitations of single-modal perception and improves the accuracy of all-weather attribute recognition. Compared with pure vision, which is easily deceived, or pure inertial perception, which suffers from serious lag, this invention innovatively constructs a cross-modal complementary mechanism of "visual feedforward + tactile feedback". By introducing a lightweight convolutional network with channel attention (SE) and fusing it with inertial frequency domain features, the estimation error of the road surface physical adhesion coefficient is controlled within ±0.05 under complex optical interference conditions such as ice and snow and water accumulation. The overall perception accuracy is improved by about 15% to 20%, effectively avoiding the failure of the feedforward model caused by single visual "misidentification".

[0122] Secondly, it eliminates the physical spatial misalignment between visual and tactile feedback, achieving "zero-delay" state fusion. Addressing the inherent physical spatiotemporal differences between forward visual anti-sighting and chassis tactile feedback, this invention implicitly introduces a vehicle kinematic curve integral model considering the front wheel steering angle, establishing a spatiotemporally aligned delay queue. Combined with a recursive Bayesian estimation algorithm, this not only completely eliminates the spatial misalignment bias between sensors but also advances the update response time of the underlying dynamic parameters by approximately 150 to 300 milliseconds, reserving ample dynamic compensation time for the underlying control.

[0123] Thirdly, it pioneers adaptive dynamic boundary constraints, significantly suppressing sideslip instability under extreme conditions. This invention breaks through the limitations of using static parameter models in traditional MPC control. Based on high-confidence perception results, it reconstructs the underlying dynamic differential equations in real time and configures the nonlinear friction circle constraint as a dynamically contracting boundary. Simulation and test data show that in extreme off-road conditions where a vehicle suddenly transitions from a high-adhesion dry asphalt road surface to a low-adhesion wet and slippery road surface at a speed of 60 km / h (e.g., the adhesion coefficient abruptly changes from 0.8 to 0.2), this invention reduces the maximum lateral tracking error of the vehicle by more than 30%, and strictly limits the peak value of the center of gravity sideslip angle within a safe threshold of ±3°. This represents a fundamental leap from "passive rescue" to "active adaptive avoidance," completely suppressing instability and tail-swing phenomena.

[0124] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of the claims of the present invention pending approval.

Claims

1. A method for intelligent vehicle trajectory tracking based on visual-touch fusion and dynamic compensation, characterized in that, Includes the following steps: Step 1: Simultaneously collect visual image data of the road surface in the pre-aiming area in front of the vehicle in the driving environment to be tested, as well as inertial vibration data of the vehicle chassis; Step 2: Extract the physical features of each modal data; wherein, a deep learning network is used to extract the road surface texture semantic features of the road surface visual image data, and frequency domain analysis is used to extract the road surface roughness features of the inertial vibration data. Step 3: Perform spatiotemporal alignment and confidence fusion of the road surface texture semantic features of the road surface visual image data and the road surface roughness features of the inertial vibration data to obtain real-time road surface dynamic parameters; Step 4: Based on the real-time road dynamics parameters, dynamically reconstruct the vehicle dynamics prediction model and dynamic boundary constraints; Step 5: Based on the reconstructed vehicle dynamics prediction model and dynamic boundary constraints, solve the optimization control problem and output vehicle actuator control commands to achieve intelligent vehicle trajectory tracking.

2. The intelligent vehicle trajectory tracking method with visual-touch fusion and dynamic compensation according to claim 1, characterized in that, In step 2, the extraction of road surface texture semantic features from visual image data using a deep learning network specifically involves: constructing a lightweight convolutional neural network that incorporates a channel attention mechanism; inputting the visual image data into the lightweight convolutional neural network; and using the convolutional layers to extract multi-channel basic road surface texture feature maps. In the lightweight convolutional neural network, the basic road surface texture feature map is recalibrated using the aforementioned channel attention mechanism, and the processing formula is as follows: ； ； ； In the formula, The feature matrix of the c-th channel in the basic road surface texture feature map. and These represent the height and width of the feature map, respectively. and These are the spatial coordinate indices in the height and width directions, respectively. and These are the weight matrices for the dimensionality reduction and dimensionality increase fully connected layers, respectively. It is the ReLU activation function. It is the Sigmoid activation function. The first one obtained after global average pooling Global space descriptor for each channel, The first time learned from the network Attention weight coefficients for each channel The output after recalibration Semantic feature map of road surface texture for each channel; The recalibrated road surface texture semantic feature map is processed using a fully connected layer. The probability distribution of road surface material categories is mapped to the category probability distribution, and the category probability distribution is weighted and calculated as the feedforward road surface reference adhesion coefficient based on the preset physical property lookup table.

3. The intelligent vehicle trajectory tracking method with visual-touch fusion and dynamic compensation according to claim 1, characterized in that, In step 2, the extraction of road surface roughness features from inertial vibration data using frequency domain analysis specifically involves: extracting a Z-axis acceleration data sequence within a preset time window. The Fast Fourier Transform (FFT) algorithm is used to transform it into a form containing the first... Complex amplitude of each frequency component The frequency domain signal is then analyzed; subsequently, based on the preset frequency band division criteria, the power spectral density (PSD) energy value of each frequency band is calculated. To construct the vibration feature vector, the relevant transformation and calculation formulas are as follows: ； ； In the formula, The total number of sampling points within the preset time window. For the time-domain sampling point index, and , It is a frequency index, and , The imaginary unit, This represents the division into low-frequency, mid-frequency, or high-frequency bands. and The first The start frequency index and end frequency index corresponding to each frequency band; The vibration feature vector containing the energy values ​​of the low-frequency, mid-frequency, and high-frequency bands Input a pre-trained support vector machine (SVM) classifier to identify the actual physical roughness of the road surface at the current wheel contact point and generate a feedback road surface reference adhesion coefficient.

4. The intelligent vehicle trajectory tracking method with visual-touch fusion and dynamic compensation according to claim 1, characterized in that, Step 3 specifically involves: performing spatiotemporal alignment and confidence fusion of the road surface texture semantic features from visual image data and the road surface roughness features from inertial vibration data to obtain real-time road surface dynamic parameters, including: Calculate the actual arc length of the travel trajectory from the current longitudinal vehicle speed and front wheel steering state to the current wheel contact surface. And solve for the time delay that satisfies the kinematic integral equation. ,in For integration time variable, Given the front wheel steering angle at the corresponding moment, the kinematic integral equation is as follows: ； Based on the time delay Establish a spatiotemporally aligned delay queue and extract from the delay queue. The feedforward road surface reference adhesion coefficient corresponding to the visual semantic features stored at each time step is used as prior information, with the current time step as the basis. The feedback road surface reference adhesion coefficient is used as likelihood information; The posterior probability after fusion is calculated using a recursive Bayesian estimation algorithm. The probability update formula is as follows: ； In the formula, The real-time pavement dynamic parameters to be estimated are... For the present The observed value at time, To until Historical observation sequence of time, The prior probability is based on historical information. The likelihood probability is based on the current feedback information. The marginal likelihood probability that serves as the normalization factor is selected as the parameter value corresponding to the maximum posterior probability as the final output, and denoted as . .

5. The intelligent vehicle trajectory tracking method with visual-touch fusion and dynamic compensation according to claim 4, characterized in that, Step 4 specifically involves: based on real-time road dynamic parameters The front and rear wheel lateral stiffnesses in the two-degree-of-freedom vehicle dynamics differential equations are updated in real time to construct a linear time-varying state space equation: ； Wherein, the state vector , Represents the state vector Derivative over time; control input Front wheel lateral stiffness in state-space equations Rear wheel lateral stiffness All are configured to be about The mapping function, whose mapping update formula is: subscript They represent the front wheel and the rear wheel, respectively. This is the tire reference lateral stiffness proportional coefficient. and These are the system state matrix and control matrix reconstructed based on the updated lateral stiffness, respectively; matrix and They are: , ； In the formula, The lateral velocity of the vehicle's center of gravity. The yaw rate is angular velocity. For longitudinal velocity, For the overall vehicle quality, To bypass moment of inertia of the shaft and These are the distances from the center of mass to the front and rear axles, respectively. For the front wheel steering angle, This refers to the vertical load on the corresponding wheel; The dynamic friction circle constraint condition based on the road surface adhesion limit is constructed and expressed by the inequality as follows: ； In the formula, d represents the front wheel or the rear wheel. For the first The longitudinal force of the wheel, For the first Lateral force of the wheel, For the first Vertical load on the wheel, when As road conditions change, the boundary range of the inequality is automatically adjusted to limit the maximum lateral and longitudinal acceleration of the vehicle.

6. The intelligent vehicle trajectory tracking method with visual-touch fusion and dynamic compensation according to claim 1, characterized in that, Step 5 specifically involves: constructing the quadratic programming objective function for Model Predictive Control (MPC). The formula is: ； In the formula, Indicates the current Always looking towards the future The predicted step size is the value of the state or input variable. To predict the step size index, This is the predicted state vector for the vehicle. The reference trajectory state vector, The control input increment represents the change in steering angle between adjacent control steps. and These are the weight matrices for the state error and the control increment, respectively. and These are the prediction time domain and the control time domain, respectively. To prevent the search for a relaxation factor that yields no feasible solution, These are the corresponding relaxation factor weighting coefficients; Under the premise of satisfying the dynamic friction circle constraint and the physical constraints of the vehicle actuator, the objective function is minimized using a solver. The optimal control increment sequence is obtained, and the first control increment in the sequence is superimposed with the actual control quantity at the previous moment and used as the vehicle actuator control command output at the current moment.

7. The intelligent vehicle trajectory tracking method with visual-touch fusion and dynamic compensation according to claim 1, characterized in that, The method further includes: Calculate the conflict entropy value between the road surface texture semantic features of the visual image data and the road surface roughness features of the inertial vibration data; if the conflict entropy value exceeds a preset safety threshold, it is determined that there is a modal conflict in the perception system. At this time, the real-time road surface dynamic parameters are forcibly set to a preset low adhesion coefficient safety value, and the system is switched to conservative control mode.

8. A smart vehicle trajectory tracking device with visual-touch fusion and dynamic compensation, used to implement the method as described in claim 1, characterized in that, include: The data acquisition module is configured to simultaneously acquire visual image data of the road surface in the pre-aiming area in front of the vehicle in the driving environment to be detected, as well as inertial vibration data of the vehicle chassis; The feature extraction module is configured to extract the road texture semantic features of the road visual image data using a deep learning network, and to extract the road roughness features of the inertial vibration data using frequency domain analysis. The fusion verification module is configured to perform spatiotemporal alignment and confidence fusion of the road surface texture semantic features and the road surface roughness features to obtain real-time road surface dynamic parameters. The control update module is configured to dynamically reconstruct the prediction model and dynamic boundary constraints of the model prediction controller based on the real-time road dynamic parameters. The execution output module is configured to solve the optimization control problem based on the reconstructed prediction model and dynamic boundary constraints, and output vehicle actuator control commands to achieve intelligent vehicle trajectory tracking.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.

Images