Vehicle control method and device

Abstract

The embodiment of the invention provides a vehicle control method and device. The control method comprises the following steps: fitting a first motion increment sequence and a second motion increment sequence by adopting a basic spline curve to obtain a first continuous time track and a second continuous time track; performing time synchronization evaluation according to the first continuous time trajectory and the second continuous time trajectory to obtain a trajectory alignment residual error; performing motion consistency evaluation according to the first continuous time track and the second continuous time track to obtain a motion consistency residual error; determining a comprehensive evaluation index according to the track alignment residual error and the motion consistency residual error; and comparing the comprehensive evaluation index with a target threshold value, and triggering a control instruction of the vehicle according to a comparison result so as to control the path planning of the vehicle. According to the embodiment of the invention, the path with the continuous curvature is generated through the B-spline control point, the prediction uncertainty is quantified, and it is ensured that the vehicle can be safely driven in a dynamic driving environment.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, specifically to a vehicle control method and device. Background Technology

[0002] In the field of intelligent driving technology for vehicles, perception and localization systems rely on the synergy of vision-inertial (VI) systems and path planning algorithms.

[0003] In related technologies, the spatiotemporal extrinsic parameter calibration of cameras and IMUs requires joint optimization of intrinsic parameters, distortion, and time offset through offline toolchains (such as Kalibr). However, relying on specific motion excitations and calibration targets makes it difficult to guarantee data quality in dynamic driving scenarios (such as high-speed lane changes and sharp turns). Online self-calibration, on the other hand, suffers from slow convergence or implicit bias due to state coupling, affecting positioning accuracy. This results in the vehicle's path planning failing to integrate real-time perception data to dynamically adjust control points, leading to response delays when facing temporary obstacles and increasing potential safety hazards for vehicle driving.

[0004] Furthermore, current evaluation processes heavily rely on ground truth sources such as motion capture systems and laser maps to calculate absolute trajectory error (ATE) and relative pose error (RPE). Existing ground truth-free methods (such as reprojection error and gravity direction consistency) suffer from limitations due to their singular metrics (only validating rotational extrinsic parameters), lack of a unified baseline (making lateral comparisons of residual norms difficult), and long-term drift monitoring blind spots. These methods cannot quantify the quality of spatiotemporal alignment, and drift in the VI system leads to distorted reference trajectories, further amplifying vehicle path planning errors and impacting safe driving. Therefore, current intelligent driving systems urgently need a systematic ground truth-free evaluation framework to quantify camera-IMU alignment quality using unified standards, addressing issues such as positioning drift and path infeasibility caused by calibration misalignment, trajectory distortion, and lack of control constraints in dynamic driving environments. Summary of the Invention

[0005] This application provides a vehicle control method and device. It generates a path with continuous curvature using B-spline control points, avoiding the abrupt curvature changes inherent in traditional straight-arc combinations and reducing steering jitter. By fitting B-spline curves based on historical trajectory points and combining lane line information, it improves curve prediction accuracy and solves the error problem of fixed curvature algorithms in curves. A multi-index fusion scoring system establishes a unified quantitative standard to achieve quantitative and comparable camera and inertial measurement unit alignment evaluation. Therefore, even in scenarios where motion capture or laser maps are unavailable, alignment quality scores can be directly obtained, supporting comparisons of different algorithms or parameter configurations and significantly reducing testing costs. Finally, a comprehensive evaluation index is generated by fusing the two types of indicators. When the index falls below a target threshold, a navigation correction command is triggered to ensure safe driving in dynamic driving environments.

[0006] The vehicle control method of this application embodiment includes: acquiring visual image stream data collected by a camera in the vehicle and measurement data from an inertial measurement unit in the vehicle, wherein the visual image stream data includes a first motion increment sequence of visual images and the measurement data includes a second motion increment sequence of the inertial measurement unit; fitting the first motion increment sequence and the second motion increment sequence with a basic spline curve to obtain a first continuous time trajectory and a second continuous time trajectory, wherein the first continuous time trajectory includes the motion trajectory of the vehicle's sensing feature points and the second continuous time trajectory includes the motion trajectory of the vehicle body; performing time synchronization evaluation based on the first continuous time trajectory and the second continuous time trajectory to obtain trajectory alignment residuals; performing motion consistency evaluation based on the first continuous time trajectory and the second continuous time trajectory to obtain motion consistency residuals; determining a comprehensive evaluation index based on the trajectory alignment residuals and the motion consistency residuals; comparing the comprehensive evaluation index with a target threshold, and triggering a control command for the vehicle based on the comparison result to control the vehicle's path planning.

[0007] In some implementations, acquiring the visual image stream data captured by the camera in the vehicle and the measurement data of the inertial measurement unit in the vehicle includes: acquiring the visual image stream data of the camera and the measurement data of the inertial measurement unit by online subscription or offline import of a preset configuration file; dynamically updating the visual image stream data and the measurement data acquired at the current moment according to a predetermined duration based on a sliding window caching module; processing the visual image stream data based on a visual odometry or simultaneous localization and mapping module to output the first motion increment sequence; and processing the measurement data through mechanized integration to output the second motion increment sequence.

[0008] In some embodiments, fitting the first motion increment sequence and the second motion increment sequence with basic spline curves to obtain the first continuous time trajectory and the second continuous time trajectory includes: performing node vector allocation based on the first motion increment sequence and the second motion increment sequence to obtain a first node vector sequence corresponding to the first motion increment sequence and a second node vector sequence corresponding to the second motion increment sequence; determining a first control node and a second control node based on the first node vector sequence, the second node vector sequence, and the basis function matrix; determining the first continuous time trajectory based on the first control node using the de Boer-Cox recursive formula; and determining the second continuous time trajectory based on the second control node using the de Boer-Cox recursive formula.

[0009] In some embodiments, the step of allocating node vectors based on the first motion increment sequence and the second motion increment sequence to obtain a first node vector sequence corresponding to the first motion increment sequence and a second node vector sequence corresponding to the second motion increment sequence includes: determining the predicted angular velocity of the vehicle in a continuous time domain based on the first motion increment sequence and the second motion increment sequence; when the value of the predicted angular velocity is greater than a first predetermined value, allocating node vectors based on chord length parameterization so that the parameter distribution of the first node vector sequence corresponding to the first motion increment sequence and the second node vector sequence corresponding to the second motion increment sequence is proportional to the displacement; when the value of the predicted angular velocity is less than a second predetermined value, allocating node vectors based on uniform parameterization so that the parameters of the first node vector sequence corresponding to the first motion increment sequence and the second node vector sequence corresponding to the second motion increment sequence are uniformly distributed.

[0010] In some embodiments, the step of performing time synchronization evaluation based on the first continuous time trajectory and the second continuous time trajectory to obtain trajectory alignment residuals includes: performing a frequency domain transformation on the phase response function of the first continuous time trajectory to obtain a first spectrum; performing a frequency domain transformation on the phase response function of the second continuous time trajectory to obtain a second spectrum; calculating the cross-spectral phase and the group delay parameters of the cross-spectral phase based on the first spectrum and the second spectrum; fitting a linear phase model within the effective frequency band of the inertial measurement unit to extract the average value and covariance of the group delay parameters; and determining the trajectory alignment residuals based on the average value and covariance of the group delay parameters.

[0011] In some embodiments, the step of performing time synchronization evaluation based on the first continuous time trajectory and the second continuous time trajectory to obtain trajectory alignment residuals includes: performing a frequency domain transformation on the phase response function of the first continuous time trajectory to obtain a first spectrum; performing a frequency domain transformation on the phase response function of the second continuous time trajectory to obtain a second spectrum; calculating the cross-spectral phase and the group delay parameter of the cross-spectral phase based on the first spectrum and the second spectrum; fitting a linear phase model within the effective frequency band of the inertial measurement unit to extract the average value of the group delay parameter; performing frequency domain coherence calculation based on the average value of the group delay parameter to determine the peak position, the peak energy corresponding to the peak position, and the time domain offset; and determining the trajectory alignment residual based on the average value of the group delay parameter, the peak energy corresponding to the peak position, and the time domain offset.

[0012] In some embodiments, the step of performing external parameter consistency evaluation based on the first continuous time trajectory and the second continuous time trajectory to obtain motion consistency residuals includes: calculating the first derivative deviation of curvature of the first continuous time trajectory and the second continuous time trajectory to generate steering dynamic residuals; calculating the second derivative deviation of acceleration of the first continuous time trajectory and the second continuous time trajectory to generate braking inertia residuals; and determining the motion consistency residuals based on the distribution mean of the steering dynamic residuals and the distribution mean of the braking inertia residuals.

[0013] In some embodiments, determining the comprehensive evaluation index based on the trajectory alignment residual and the motion consistency residual includes: when the vehicle speed is outside a first predetermined range, fusing a first weighting coefficient, the trajectory alignment residual, and the motion consistency residual to determine the comprehensive evaluation index; when the vehicle's steering angular velocity is outside a second predetermined range, fusing a second weighting coefficient, the trajectory alignment residual, and the motion consistency residual to determine the comprehensive evaluation index; and when the vehicle's acceleration is outside a third predetermined range, fusing a third weighting coefficient, the trajectory alignment residual, and the motion consistency residual to determine the comprehensive evaluation index.

[0014] In some embodiments, the vehicle control method further includes: providing a visualization interface that can be configured to generate a formatted report; and displaying, based on data update signals provided by the vehicle, real-time updated visual image stream data and measurement data, trajectory alignment residuals, motion consistency residuals, comprehensive evaluation indicators, and the vehicle's driving status.

[0015] The vehicle control device of this application embodiment includes a data acquisition module configured to acquire visual image stream data captured by a camera in the vehicle and measurement data from an inertial measurement unit in the vehicle. The visual image stream data includes a first motion increment sequence of the visual images, and the measurement data includes a second motion increment sequence from the inertial measurement unit. A fitting module is configured to fit the first motion increment sequence and the second motion increment sequence using a basic spline curve to obtain a first continuous time trajectory and a second continuous time trajectory. The first continuous time trajectory includes the motion trajectory of the vehicle's sensing feature points, and the second continuous time trajectory includes the motion trajectory of the vehicle's own sensor feature points. The system includes a motion trajectory assessment module, configured to perform time synchronization assessment based on the first continuous time trajectory and the second continuous time trajectory to obtain trajectory alignment residuals; an extrinsic parameter consistency assessment module, configured to perform extrinsic parameter consistency assessment based on the first continuous time trajectory and the second continuous time trajectory to obtain motion consistency residuals; a comprehensive assessment module, configured to determine a comprehensive assessment index based on the trajectory alignment residuals and the motion consistency residuals; and a signal triggering module, configured to compare the comprehensive assessment index with a target threshold and trigger the vehicle's control command based on the comparison result to control the vehicle's path planning.

[0016] This application's embodiments generate curvature-continuous paths using B-spline control points, avoiding the abrupt curvature changes inherent in traditional straight-arc combinations and reducing steering jitter. It fits B-spline curves based on historical trajectory points and combines lane line information to improve curve prediction accuracy, resolving the error problem of fixed curvature algorithms on curves. A multi-index fusion scoring system establishes a unified quantitative standard to achieve quantitative and comparable camera and inertial measurement unit alignment evaluation. Therefore, even in scenarios where motion capture or laser maps are unavailable, alignment quality scores can be directly obtained, supporting comparisons of different algorithms or parameter configurations and significantly reducing testing costs. Finally, a comprehensive evaluation index is generated by fusing the two types of indicators. When the index falls below a target threshold, a navigation correction command is triggered to ensure safe driving in dynamic driving environments. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the vehicle control method provided in an embodiment of this application.

[0019] Figure 2This is a schematic diagram illustrating the process of dynamically updating the data stream using a sliding window caching module provided in an embodiment of this application.

[0020] Figure 3 This is a schematic diagram of the basic spline curve fitting process provided for the embodiments of this application.

[0021] Figure 4 This is a flowchart illustrating the dynamic adjustment of continuous data trajectories provided in an embodiment of this application.

[0022] Figure 5 This is a schematic diagram of a time synchronization evaluation process provided for an embodiment of this application.

[0023] Figure 6 Another flowchart illustrating the time synchronization evaluation provided for embodiments of this application.

[0024] Figure 7 This is a schematic diagram of the process for evaluating the consistency of external parameters provided in the embodiments of this application.

[0025] Figure 8 This is a schematic diagram of the vehicle's control device. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] This application provides a vehicle control method and a vehicle control device.

[0028] The solutions provided in this application relate to the field of vehicle technology, and are specifically illustrated through the following embodiments. Detailed descriptions are provided below. It should be noted that the order of description of the following embodiments is not intended to limit the priority of the embodiments.

[0029] The vehicle control method provided by the exemplary embodiments of this application will be described below with reference to the accompanying drawings and the application scenarios described above. It should be noted that the application scenarios described above are only shown for the purpose of understanding the principles of this application, and the embodiments of this application are not limited in any way.

[0030] Figure 1 This is a schematic diagram of a vehicle control device provided in an embodiment of this application. Figure 1 As shown, the vehicle control method provided in this application includes: Step 010: Acquire the visual image stream data captured by the camera in the vehicle and the measurement data of the inertial measurement unit in the vehicle. The visual image stream data includes the first motion increment sequence of the visual image, and the measurement data includes the second motion increment sequence of the inertial measurement unit. Step 020: Fit the first motion increment sequence and the second motion increment sequence using basic spline curves to obtain the first continuous time trajectory and the second continuous time trajectory. The first continuous time trajectory includes the motion trajectory of the vehicle sensing feature points, and the second continuous time trajectory includes the motion trajectory of the vehicle body. Step 030: Perform time synchronization evaluation based on the first and second continuous time trajectories to obtain the trajectory alignment residual; Step 040: Based on the first continuous time trajectory and the second continuous time trajectory, perform motion consistency assessment to obtain motion consistency residuals; Step 050: Determine the comprehensive evaluation index based on the trajectory alignment residual and motion consistency residual; Step 060: Compare the comprehensive evaluation index with the target threshold, and trigger the vehicle control command based on the comparison result to control the vehicle's path planning.

[0031] Specifically, B-spline curves, with their piecewise polynomial properties and high-order continuity, have become a core tool for path planning and state monitoring in intelligent driving. B-splines generate curvature-continuous paths (such as in automatic parking scenarios) through control points, avoiding the abrupt curvature changes of traditional straight-line-circular-arc combinations, reducing steering jitter. By fitting B-spline curves based on historical trajectory points and combining lane line information, the accuracy of curve prediction is improved, solving the error problem of fixed curvature algorithms in curves.

[0032] The time synchronization index can be used to quantify the timestamp deviation between the visual image stream and the IMU data, and the spatial transformation relationship between the camera and the IMU can be quantified through external parameter consistency evaluation.

[0033] By leveraging the continuous second derivative of B-splines, the path curvature change rate is calculated in real time to identify abnormal steering behavior. The resulting motion consistency residuals can be used to quantize the spatial transformation relationship between the camera and the IMU to achieve vehicle trajectory planning and positioning.

[0034] The comprehensive evaluation index can normalize the trajectory alignment residual and motion consistency residual, replacing external references with relevant parameters from these residuals. This allows for alignment quality assessment without requiring motion capture or laser maps, thus enabling online monitoring of vehicle navigation status. Furthermore, the comprehensive evaluation index avoids the failure of a single index, improving the reliability of the evaluation method.

[0035] Thus, by using basic spline curve fitting to fit the data from the visual sensor and the inertial measurement unit (IMU), continuous data trajectories defined in the continuous time domain replace discrete data. This ensures that the vehicle can process paths with continuous curvature generated by the sensors (IMU, camera) in real time, quantifying the predictive uncertainty of path planning, and is suitable for intelligent driving scenarios. A multi-index fusion scoring system establishes a unified quantitative standard to achieve quantitative and comparable camera and inertial measurement unit alignment evaluation. Therefore, even in scenarios where motion capture or laser maps are unavailable, alignment quality scores can be directly obtained, supporting comparisons of different algorithms or parameter configurations and significantly reducing testing costs. Finally, the two types of indicators are fused to generate a comprehensive evaluation index. When the index falls below a target threshold, a calibration command is triggered to ensure safe driving in dynamic driving environments.

[0036] Figure 2 This is a schematic diagram illustrating the process of dynamically updating the data stream using a sliding window caching module provided in an embodiment of this application. Figure 2 As shown, in some embodiments, step 010 above includes: Step 011: Obtain the camera's visual image stream data and the inertial measurement unit's measurement data by subscribing online or importing the preset configuration file offline; Step 012: Based on the sliding window caching module, dynamically update the visual image stream data and measurement data acquired at the current moment according to a predetermined duration; Step 013: Process the visual image stream data based on the visual odometry or simultaneous localization and mapping module, and output the first motion increment sequence; Step 014: Process the measurement data through mechanized integration and output the second motion increment sequence.

[0037] Specifically, the input interface for acquiring data streams supports both online and offline modes. In online mode, the ROS2 framework is plug-and-play, allowing for the construction of modular evaluation nodes that can be directly deployed within the system. It can directly connect to existing robots and autonomous driving systems for real-time data stream analysis, and flexibly configure and obtain results through ROS parameters and topic interfaces. In offline mode, real-time data streams can be simulated by parsing CSV files (timestamps, image paths, and raw IMU data). The sliding window caching module can save the most recent N seconds (configurable) of image and IMU data, ensuring the timeliness of the analysis data. It can dynamically output time synchronization and extrinsic parameter quality scores online, making it suitable for long-duration tasks.

[0038] Visual odometry (VO) or simultaneous localization and mapping (SLAM) modules utilize a continuous stream of images captured by a camera. Through feature extraction, feature matching, and tracking, they calculate the relative pose changes between adjacent keyframes, outputting a first motion increment sequence. The raw data from the inertial measurement unit (IMU) is then processed, and a second motion increment sequence is output through time integration.

[0039] Thus, this application embodiment introduces a real-time sliding window analysis framework, embedding time synchronization and external parameter consistency evaluation into the sliding window mechanism, calculating and updating indicators in real time, realizing online quality monitoring and dynamic diagnosis, so that time synchronization and external parameter quality can be dynamically tracked during system operation, and time offset changes or external parameter inaccuracies can be detected in real time during long-term unmanned vehicle tasks, thereby improving system stability and maintenance efficiency.

[0040] Figure 3 This is a schematic diagram illustrating the process of fitting a basic spline curve for an embodiment of this application. Figure 3 As shown, step 020 above includes: Step 021: Based on the first motion increment sequence and the second motion increment sequence, perform node vector allocation to obtain the first node vector sequence corresponding to the first motion increment sequence and the second node vector sequence corresponding to the second motion increment sequence; Step 022: Determine the first control node and the second control node based on the first node vector sequence, the second node vector sequence, and the basis function matrix; Step 023: Based on the first control node, determine the first continuous time trajectory using the de Boer-Cox recursive formula; Step 024: Based on the second control node, determine the second continuous time trajectory using the de Boer-Cox recursive formula.

[0041] Specifically, in the first motion increment sequence, the displacements of feature points are aligned according to the timestamps to generate a temporal motion sequence {Δp1vis,1,Δp1vis,2,…,Δp1vis,m}.

[0042] In the second motion increment sequence, the displacements of the feature points are aligned according to the timestamps to generate the temporal motion sequence {Δp2vis,1,Δp2vis,2,…,Δp2vis,m}.

[0043] Let the number of control points be n+1, the B-spline order be p (degree p), and the number of internal nodes be m−n. The specific formula for calculating the number of internal nodes is as follows: The starting node is repeated p+1 times: U0=U1=⋯=Up=0 Intermediate nodes: Up+1=t1,…,Un=tm−n The terminator repeats p+1 times: Un+1=⋯=Un+p+1=1 A complete node vector can be represented as: Construct a system of linear equations, assuming the number of control points is n+1, and the basis function matrix N consists of Ni,3(uk). Solve the system: Matrix form: NQ=P, where P is the sequence of motion increments, and the overdetermined equations are solved using QR decomposition or SVD decomposition.

[0044] Third-order B-spline equation: The corresponding continuous-time trajectory is obtained by calculating using the de Boer-Cox recurrence formula: Figure 4 This is a schematic diagram illustrating the process of dynamically adjusting the continuous data trajectory in basic spline curve fitting, as provided in the embodiments of this application. Figure 4 As shown, step 021 above also includes: Step 0211: Determine the predicted angular velocity of the vehicle in the continuous time domain based on the first motion increment sequence and the second motion increment sequence; Step 0212: If the predicted angular velocity is greater than the first predetermined value, perform node vector allocation based on chord length parameterization so that the parameter distribution of the first node vector sequence corresponding to the first motion increment sequence and the second node vector sequence corresponding to the second motion increment sequence is proportional to the displacement. Step 0213: If the predicted angular velocity is less than the second predetermined value, perform node vector allocation based on uniform parameterization so that the parameters of the first node vector sequence corresponding to the first motion increment sequence and the second node vector sequence corresponding to the second motion increment sequence are uniformly distributed.

[0045] Specifically, when the predicted angular velocity is greater than a first predetermined value, node vector allocation is performed based on chord length parameterization to improve the fitting accuracy for rapid head turns. For example, when the predicted angular velocity ω > 2 rad / s, the fitting accuracy for rapid head turns is improved to adapt to intelligent driving scenarios involving emergency vehicle turns.

[0046] In the chord length parameterization process, the cumulative chord length is first calculated. Given a sequence of type value points P0, P1, ..., Pm, initialization is performed: d0 = 0. Iterative calculation is then performed: dk = dk−1 + ||Pk−Pk−1|| (k = 1, 2, ..., m). Normalization generates a parameter sequence, with the parameter uk allocated proportionally to the cumulative chord length. When the predicted angular velocity is less than a first predetermined value, node vector allocation is performed based on uniform parameterization, and the fitting accuracy of the continuous time trajectory is reduced. For example, when ω < 0.8 rad / s, the jitter caused by sensor noise is suppressed, and the fitting accuracy of the continuous time trajectory is reduced to adapt to the intelligent driving scenario where the vehicle speed changes smoothly.

[0047] For a k-th order B-spline curve, the parameter u is uniformly distributed on the nodal vector U={0,1,2,…,n}, and the curve equation is: Figure 5 This is a schematic diagram of a time synchronization evaluation process provided for an embodiment of this application. Figure 5 As shown, in some embodiments, step 030 above includes: Step 0311: Perform frequency domain transformation on the phase response function of the first continuous time trajectory to obtain the first spectrum; Step 0312: Perform a frequency domain transformation on the phase response function of the second continuous time trajectory to obtain the second spectrum; Step 0313: Calculate the cross-spectral phase and group delay parameters of the cross-spectral phase based on the first and second spectra: Step 0314: Fit a linear phase model within the effective frequency band of the inertial measurement unit and extract the average value and covariance of the group delay parameters; Step 0315: Determine the trajectory alignment residual based on the average value and covariance of the group delay parameters.

[0048] Specifically, the phase response function of the first continuous-time trajectory is transformed in the frequency domain to obtain the first spectrum Sv(ω). The phase response function of the second continuous-time trajectory is transformed in the frequency domain to obtain the second spectrum Si(ω). The cross-spectral phase ϕ(ω) is calculated as arg(Sv(ω)⋅Si∗(ω)), where Si∗(ω) is the complex conjugate of the IMU spectrum.

[0049] The group delay parameter is calculated using the phase slope method: τg(ω) = −dϕ(ω) / dω. A linear phase model is fitted within the effective frequency band [ωmin, ωmax] of the inertial measurement unit, and the mean group delay and slope variance are extracted. The mean group delay quantifies the overall delay estimate; a larger mean group delay indicates poorer time synchronization between the visual image stream data and the measurement data, and a smaller trajectory alignment residual. The slope variance quantifies the phase linearity within the frequency band; a larger slope variance indicates better time synchronization between the visual image stream data and the measurement data, and a larger trajectory alignment residual.

[0050] Thus, the average value and covariance of the group delay parameters obtained by coherence analysis using the group delay / phase slope method can quantify the trajectory alignment residuals.

[0051] Figure 6 Another schematic diagram illustrating the time synchronization evaluation provided for embodiments of this application. For example... Figure 6 As shown, in some embodiments, step 030 above further includes: Step 0321: Perform frequency domain transformation on the phase response function of the first continuous time trajectory to obtain the first spectrum; Step 0322: Perform a frequency domain transformation on the phase response function of the second continuous-time trajectory to obtain the second spectrum; Step 0323: Calculate the cross-spectral phase and group delay parameters of the cross-spectral phase based on the first and second spectra: Step 0324: Fit a linear phase model within the effective frequency band of the inertial measurement unit and extract the average value of the group delay parameter; Step 0325: Perform frequency domain coherence calculation based on the average value of the group delay parameters to determine the peak position, the peak energy corresponding to the peak position, and the time domain offset; Step 0326: Determine the trajectory alignment residual based on the average value of the group delay parameters, the peak energy corresponding to the peak position, and the time domain offset.

[0052] Specifically, the logarithmic mapping is obtained from the relative rotation matrices of adjacent camera frames. By performing a sliding cross-correlation with the IMU gyroscope measurement ωimu, we can find the maximum peak position Δt^=argmaxτCCF(τ) and the peak energy Epeak=max(CCF).

[0053] Peak energy can be used as a cross-correlation peak parameter to quantify trajectory alignment residuals. The larger the peak energy, the better the time synchronization between visual image stream data and measurement data, and the larger the trajectory alignment residual.

[0054] The time-domain offset corresponding to the peak position is t0. The absolute difference between the time-domain offset and the mean group delay must be lower than a preset tolerance threshold; otherwise, a time synchronization anomaly flag is triggered. The smaller the absolute difference between the time-domain offset and the mean group delay, the better the time synchronization between the visual image stream data and the measurement data, and the larger the trajectory alignment residual.

[0055] Thus, the peak energy obtained by cross-correlation analysis using the group delay / phase slope method can quantify the trajectory alignment residual.

[0056] Traditional time synchronization assessments rely on hardware triggers, external clocks, or ground truth trajectories; they cannot be quantified when these conditions are missing.

[0057] In the implementation of step 030 above, by using the visual angular velocity and group delay / phase slope method + IMU gyroscope cross-correlation + maximum coherence (MSC) analysis, synchronization deviation can be quickly detected even without hardware synchronization or external time reference, thus solving the common problem of missing time reference in existing engineering deployments.

[0058] Figure 7 This is a schematic diagram illustrating the process of external parameter consistency evaluation provided in an embodiment of this application. Figure 7 As shown, in some embodiments, step 040 above includes: Step 041: Calculate the first derivative deviation of curvature of the first continuous time trajectory and the second continuous time trajectory to generate the steering dynamic residual; Step 042: Calculate the deviation of the second derivative of acceleration between the first continuous time trajectory and the second continuous time trajectory to generate braking inertia residual; Step 043: Determine the motion consistency residual based on the mean distribution of the steering dynamic residual and the mean distribution of the braking inertia residual.

[0059] Specifically, curvature describes the degree of trajectory bending, and the first derivative of curvature reflects the smoothness of steering changes (such as vehicle steering vibration). Here, C′(u) and C′′(u) are the first and second derivatives of the B-spline curve, respectively, and the specific formulas are as follows: The second derivative of acceleration *a* reflects the degree of abrupt change in motion (such as rapid acceleration or deceleration), where *p* is the order of the B-spline, *Pi* is the acceleration of the B-spline curve at the control point, and *a(u)* = *C′′(u)*. Its derivative is calculated using the third-order control point difference quotient, and the formula is as follows: The larger the dynamic residual, the worse the consistency of motion between the first and second continuous time trajectories. Similarly, the larger the braking inertia residual, the worse the consistency of motion between the first and second continuous time trajectories. Covariance can normalize the dynamic and braking inertia residuals, avoiding excessive dependence on a single residual.

[0060] In this way, dynamic residuals, braking inertia residuals, and covariances can be calculated from multiple sets of data. The mean values ​​of the steering dynamic residuals and braking inertia residuals corresponding to these multiple sets of data can also be calculated, quantifying the motion consistency residuals. The greater the deviation of the residual distribution from the ideal zero mean, the smaller the motion consistency residual.

[0061] In the implementation of step 040 above, residual decomposition provides a direct and quantitative evaluation of extrinsic parameters. Therefore, extrinsic parameter rotational drift or instability can be detected earlier, preventing misuse that could lead to a decrease in overall positioning accuracy.

[0062] In some embodiments, step 50 above includes: Step 051: When the vehicle speed is outside the first predetermined range, the first weighting coefficient, trajectory alignment residual, and motion consistency residual are fused to determine the comprehensive evaluation index; Step 052: When the angular velocity of the vehicle's steering is outside the second predetermined range, the second weighting coefficient, trajectory alignment residual, and motion consistency residual are fused to determine the comprehensive evaluation index; Step 053: When the vehicle's acceleration is outside the third predetermined interval, the third weighting coefficient, trajectory alignment residual, and motion consistency residual are fused to determine the comprehensive evaluation index.

[0063] Specifically, the weighting coefficients can be adjusted in real time based on the vehicle's driving conditions. If the vehicle speed is outside the first predetermined range, indicating an abnormal speed, the value of the first weighting coefficient is increased to prioritize high-speed stability and strictly control trajectory deviation. If the vehicle's steering angular velocity is outside the second predetermined range, indicating an abnormal steering, the value of the second weighting coefficient is decreased, prioritizing the vehicle's maneuverability and allowing for temporary path deviations. If the vehicle's acceleration is outside the third predetermined range, the value of the third weighting coefficient is increased, prioritizing motion consistency and suppressing sensor noise accumulation.

[0064] In some implementations, step 50 includes: weighting and summing at least one parameter in the trajectory alignment residual and at least one parameter in the motion consistency residual to determine a comprehensive evaluation index.

[0065] Specifically, the selection of indicators for constructing a comprehensive evaluation system using a multi-indicator fusion mechanism includes one or more parameters such as time synchronization indicators and external parameter consistency indicators.

[0066] The parameters for the time synchronization index include the selected time offset (through cross-correlation) and synchronization confidence (such as signal coherence MSC). The parameters for the extrinsic consistency index include the gravity angle (gravity alignment error), the mean reprojection error (spatial pose deviation), and the variance of the pre-integrated residual (motion continuity). The parameters are linearly weighted according to the configured weights wi to form the total score. As a comprehensive evaluation indicator.

[0067] Step 50 above can further construct a hierarchical sub-scoring system, wherein: The time-domain sub-scoring is generated by weighting the peak position, peak energy, and event delay distribution variance of the fused trajectory alignment residuals according to preset weights, serving as the first evaluation value. The frequency-domain sub-scoring is generated by weighting the mean group delay, slope variance, and frequency coherence confidence of the fused trajectory alignment residuals according to preset weights, serving as the second evaluation value. The geometric-domain sub-scoring is generated by weighting the mean rotation geodesic distance and position residual variance of the fused motion consistency residuals according to preset weights, serving as the third evaluation value.

[0068] Layered fusion ensures the reliability of time-domain focused delay estimation, frequency-domain quantization ensures phase linearity and frequency band consistency, and geometric domain unifies rotation / translation / gravity constraints, making it suitable for dynamic working scenarios so that the relevant weight ratios can be dynamically adjusted according to the actual working conditions of the vehicle.

[0069] In some implementations, step 60 above can determine the evaluation result of the camera and inertial measurement unit alignment by setting a three-level threshold: if Statal ≥ 0.85 → qualified (alignment accuracy meets real-time fusion requirements), 0.6 ≤ Statal < 0.85 → warning (calibration parameters need to be monitored or optimized), Statal < 0.6 → unqualified (external parameters or time synchronization failure, triggering recalibration).

[0070] In some embodiments, the vehicle control method further includes: It provides a visual interface that can be configured to generate formatted reports; Based on the data update signals provided by the vehicle, the visualization interface displays real-time updated visual image stream data and measurement data, trajectory alignment residuals, motion consistency residuals, comprehensive evaluation indicators, and the vehicle's calibration status.

[0071] Specifically, this application provides a unified automated report generation (JSON metrics + charts), which outputs cross-correlation curves, residual distributions, and score summaries with a single click, compatible with both offline and online scenarios. This significantly simplifies the engineering debugging process, reduces manual analysis time, and improves efficiency and reusability.

[0072] Users can monitor the vehicle's data updates in real time through a visual interface in the background, enabling online monitoring of the vehicle's navigation.

[0073] Figure 8 This is a structural schematic diagram of the vehicle control device 100. (Example) Figure 8 As shown, the vehicle control device 100 includes: The data acquisition module 110 is configured to acquire visual image stream data captured by a camera in the vehicle and measurement data from an inertial measurement unit in the vehicle. The visual image stream data includes a first motion increment sequence of the visual image, and the measurement data includes a second motion increment sequence of the inertial measurement unit. The fitting module 120 is configured to fit the first motion increment sequence and the second motion increment sequence using a basic spline curve to obtain a first continuous time trajectory and a second continuous time trajectory. The first continuous time trajectory includes the motion trajectory of the vehicle sensing feature points, and the second continuous time trajectory includes the motion trajectory of the vehicle body. The time synchronization evaluation module 130 is configured to perform time synchronization evaluation based on the first continuous time trajectory and the second continuous time trajectory to obtain the trajectory alignment residual. The external parameter consistency assessment module 140 is configured to perform an external parameter consistency assessment based on the first continuous time trajectory and the second continuous time trajectory to obtain the motion consistency residual. The comprehensive evaluation module 150 is configured to determine comprehensive evaluation indicators based on trajectory alignment residuals and motion consistency residuals; The signal triggering module 160 is configured to compare the comprehensive evaluation index with the target threshold and trigger the vehicle control command based on the comparison result to control the vehicle's path planning.

[0074] The vehicle control device provided in this application embodiment also includes a computer device. For example, the vehicle control method in this application embodiment can be executed by the computer device.

[0075] The computer device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments, which will not be repeated here.

[0076] In some embodiments, the processor may invoke software programs and modules stored in memory to execute the vehicle control method described above.

[0077] In some embodiments, the computer device may be integrated into a terminal or server that has storage and a processor and thus computing power, or the computer device may be the terminal or server.

[0078] It should be understood that the processor in the embodiments of this application may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0079] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0080] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0081] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0082] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0083] In addition, the functional units in the embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0084] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for controlling a vehicle, characterized in that, include: The system acquires visual image stream data captured by a camera in the vehicle and measurement data from an inertial measurement unit in the vehicle. The visual image stream data includes a first motion increment sequence of the visual images, and the measurement data includes a second motion increment sequence of the inertial measurement unit. The first motion increment sequence and the second motion increment sequence are fitted with basic spline curves to obtain the first continuous time trajectory and the second continuous time trajectory. The first continuous time trajectory includes the motion trajectory of the vehicle sensing feature points, and the second continuous time trajectory includes the motion trajectory of the vehicle body. Based on the first continuous time trajectory and the second continuous time trajectory, a time synchronization assessment is performed to obtain the trajectory alignment residual; Based on the first continuous time trajectory and the second continuous time trajectory, motion consistency assessment is performed to obtain motion consistency residuals; Based on the trajectory alignment residual and the motion consistency residual, a comprehensive evaluation index is determined; The comprehensive evaluation index is compared with the target threshold, and the control command of the vehicle is triggered based on the comparison result to control the vehicle's path planning.

2. The vehicle control method as described in claim 1, characterized in that, The acquisition of visual image stream data captured by the camera in the vehicle and measurement data from the inertial measurement unit in the vehicle includes: The camera's visual image stream data and the inertial measurement unit's measurement data can be obtained by subscribing online or importing preset configuration files offline. Based on the sliding window caching module, the visual image stream data and the measurement data acquired at the current moment are dynamically updated according to a predetermined duration; The visual image stream data is processed based on a visual odometry or simultaneous localization and mapping module, and the first motion increment sequence is output. The measurement data is processed by mechanized integration to output the second motion increment sequence.

3. The vehicle control method as described in claim 1, characterized in that, The step of fitting the first motion increment sequence and the second motion increment sequence with a basic spline curve to obtain the first continuous time trajectory and the second continuous time trajectory includes: Based on the first motion increment sequence and the second motion increment sequence, node vector allocation is performed to obtain the first node vector sequence corresponding to the first motion increment sequence and the second node vector sequence corresponding to the second motion increment sequence; The first control node and the second control node are determined based on the first node vector sequence, the second node vector sequence, and the basis function matrix. Based on the first control node, the first continuous time trajectory is determined using the de Boer-Cox recursive formula; The second continuous time trajectory is determined based on the second control node using the de Boer-Cox recursive formula.

4. The vehicle control method as described in claim 3, characterized in that, The step of allocating node vectors based on the first motion increment sequence and the second motion increment sequence to obtain a first node vector sequence corresponding to the first motion increment sequence and a second node vector sequence corresponding to the second motion increment sequence includes: Based on the first motion increment sequence and the second motion increment sequence, the predicted angular velocity of the vehicle in the continuous time domain is determined; If the predicted angular velocity is greater than a first predetermined value, node vector allocation is performed based on chord length parameterization so that the parameter distribution of the first node vector sequence corresponding to the first motion increment sequence and the second node vector sequence corresponding to the second motion increment sequence is proportional to the displacement. If the predicted angular velocity is less than a second predetermined value, node vector allocation is performed based on uniform parameterization to ensure that the parameters of the first node vector sequence corresponding to the first motion increment sequence and the second node vector sequence corresponding to the second motion increment sequence are uniformly distributed.

5. The vehicle control method as described in claim 1, characterized in that, The step of performing time synchronization evaluation based on the first continuous time trajectory and the second continuous time trajectory to obtain trajectory alignment residuals includes: The first spectrum is obtained by performing a frequency domain transformation on the phase response function of the first continuous time trajectory; The second continuous-time trajectory phase response function is transformed in the frequency domain to obtain the second spectrum; Calculate the cross-spectral phase and the group delay parameter of the cross-spectral phase based on the first spectrum and the second spectrum: A linear phase model is fitted within the effective frequency band of the inertial measurement unit, and the average value and covariance of the group delay parameters are extracted. The trajectory alignment residual is determined based on the average value and covariance of the group delay parameters.

6. The vehicle control method as described in claim 1, characterized in that, The step of performing time synchronization evaluation based on the first continuous time trajectory and the second continuous time trajectory to obtain trajectory alignment residuals includes: The first spectrum is obtained by performing a frequency domain transformation on the phase response function of the first continuous time trajectory; The second continuous-time trajectory phase response function is transformed in the frequency domain to obtain the second spectrum; Calculate the cross-spectral phase and the group delay parameter of the cross-spectral phase based on the first spectrum and the second spectrum: A linear phase model is fitted within the effective frequency band of the inertial measurement unit, and the average value of the group delay parameter is extracted. Frequency domain coherence is calculated based on the average value of the group delay parameters to determine the peak position, the peak energy corresponding to the peak position, and the time domain offset. The trajectory alignment residual is determined based on the average value of the group delay parameters, the peak energy corresponding to the peak position, and the time domain offset.

7. The vehicle control method as described in claim 1, characterized in that, The step of performing an external parameter consistency assessment based on the first continuous time trajectory and the second continuous time trajectory to obtain motion consistency residuals includes: Calculate the first-order derivative deviation of curvature between the first continuous time trajectory and the second continuous time trajectory to generate the steering dynamic residual; Calculate the deviation of the second derivative of acceleration between the first continuous time trajectory and the second continuous time trajectory to generate braking inertia residual; The motion consistency residual is determined based on the mean distribution of the steering dynamic residual and the mean distribution of the braking inertia residual.

8. The vehicle control method as described in claim 1, characterized in that, The determination of comprehensive evaluation indicators based on the trajectory alignment residual and the motion consistency residual includes: When the vehicle speed is outside the first predetermined range, the first weighting coefficient, the trajectory alignment residual, and the motion consistency residual are fused to determine a comprehensive evaluation index; When the angular velocity of the vehicle's steering is outside the second predetermined range, the second weighting coefficient, the trajectory alignment residual, and the motion consistency residual are fused to determine a comprehensive evaluation index; When the vehicle's acceleration is outside the third predetermined interval, the third weighting coefficient, the trajectory alignment residual, and the motion consistency residual are fused to determine a comprehensive evaluation index.

9. The vehicle control method according to any one of claims 1-8, characterized in that, The vehicle control method also includes: Provides a visualization interface that can be configured to generate formatted reports; Based on the data update signals provided by the vehicle, the visualization interface displays the real-time updated visual image stream data and measurement data, the trajectory alignment residual, the motion consistency residual, the comprehensive evaluation index, and the vehicle's driving status.

10. A vehicle control device, characterized in that, include: The data acquisition module is configured to acquire visual image stream data captured by a camera in the vehicle and measurement data from an inertial measurement unit in the vehicle. The visual image stream data includes a first motion increment sequence of the visual images, and the measurement data includes a second motion increment sequence of the inertial measurement unit. The fitting module is configured to fit the first motion increment sequence and the second motion increment sequence using a basic spline curve to obtain the first continuous time trajectory and the second continuous time trajectory. The first continuous time trajectory includes the motion trajectory of the vehicle sensing feature points, and the second continuous time trajectory includes the motion trajectory of the vehicle body. The time synchronization evaluation module is configured to perform time synchronization evaluation based on the first continuous time trajectory and the second continuous time trajectory to obtain trajectory alignment residuals; The external parameter consistency evaluation module is configured to perform an external parameter consistency evaluation based on the first continuous time trajectory and the second continuous time trajectory to obtain the motion consistency residual. The comprehensive evaluation module is configured to determine a comprehensive evaluation index based on the trajectory alignment residual and the motion consistency residual; The signal triggering module is configured to compare the comprehensive evaluation index with the target threshold and trigger the control command of the vehicle based on the comparison result to control the vehicle's path planning.

Images