Vehicle control method and device, vehicle and storage medium

Abstract

The invention relates to the technical field of vehicle control, and discloses a vehicle control method and device, a vehicle and a storage medium, and the method comprises the steps: carrying out the fusion of obtained inertial measurement data and foresight vision data, obtaining fusion perception data, inputting the fusion perception data into a multi-modal neural network model, and obtaining a multi-modal neural network model; obtaining an environment perception result output by the model, dividing a current drivable area in the drivable area semantic map into multiple units, evaluating a road coupling risk index value for each unit based on the environment perception result and inertial measurement data, and obtaining a road coupling risk index value for each unit based on a road coupling risk index value corresponding to a target unit into which a vehicle is about to drive; and a corresponding vehicle control strategy is generated to control the vehicle to run, so that the risk area where the vehicle is about to run is recognized in advance, pre-judgment and control are carried out in advance, and the potential safety hazard of driving is greatly reduced.

Description

A vehicle control method, device, vehicle, and storage medium Technical Field

[0001] This invention relates to the field of vehicle control technology, specifically to a vehicle control method, device, vehicle, and storage medium. Background Technology

[0002] With the rapid development of intelligent driving technology, vehicle safety control systems are evolving from single-sensor perception to a multi-modal perception-risk assessment-intelligent control approach. Currently, forward-facing cameras have become the core visual sensor for Advanced Driver Assistance Systems (ADAS), and inertial measurement units (IMUs) provide crucial support for vehicle dynamic state perception. However, despite significant progress in environmental perception technology and vehicle speed control systems, the multi-modal vehicle driving data collected by each system has not been effectively integrated and utilized, making it impossible to perform precise vehicle speed control based on comprehensive environmental conditions. Furthermore, current vehicle safety control systems largely rely on preset static rules (such as deceleration in curves and tunnels), with the system making judgments based on simple rules. These rules are mostly based on idealized scenario assumptions and do not consider the dynamic impact of real-time terrain features on vehicle control. Consequently, they cannot perform precise vehicle control based on real-time risk terrain features, affecting driving experience and safety. Summary of the Invention

[0003] This invention provides a vehicle control method, device, vehicle, and storage medium to solve the problems of inability to perform precise vehicle speed control based on comprehensive environmental conditions and inability to perform precise vehicle control based on real-time risk characteristics.

[0004] In a first aspect, the present invention provides a vehicle control method, the method comprising: acquiring inertial measurement data and forward-looking visual data of a vehicle; fusing the inertial measurement data and the forward-looking visual data to obtain fused perception data; inputting the fused perception data into a multimodal neural network model to obtain an environmental perception result output by the model, the environmental perception result including at least a drivable area semantic map; dividing the currently drivable area in the drivable area semantic map into multiple risk calculation units, and evaluating a road coupling risk index value for each risk calculation unit based on the environmental perception result and the inertial measurement data, the road coupling risk index value being used to quantify the driving risk level of a road under the coupling effect of multiple factors; generating a corresponding vehicle control strategy based on the road coupling risk index value corresponding to the target risk calculation unit into which the vehicle is about to enter, and controlling the vehicle's driving based on the vehicle control strategy.

[0005] The vehicle control method provided by this invention fuses acquired inertial measurement data and forward-looking visual data to obtain fused perception data, achieving coordinated linkage between environmental and vehicle data. This provides high-quality data support for subsequent precise perception and control, and is adaptable to complex driving scenarios. The fused perception data is input into a multimodal neural network model to obtain the environmental perception results output by the model. The environmental perception results include a drivable area semantic map, which divides the currently drivable area into multiple risk calculation units. Based on the environmental perception results and inertial measurement data, the road coupling risk index value is evaluated for each risk calculation unit. Based on the road coupling risk index value corresponding to the target risk calculation unit that the vehicle is about to enter, a corresponding vehicle control strategy is generated to control the vehicle's movement. This method enables early identification of risk areas that the vehicle is about to enter, allowing for advance prediction and control, significantly reducing driving safety hazards. Furthermore, it fully considers the coupling effects of various factors such as real-time terrain features and obstacle states to determine the risk level of the target risk unit and generate differentiated vehicle control strategies. This results in higher control accuracy and adaptability, avoiding control errors caused by the disconnect between static rules and actual complex scenarios, and improving driving experience and safety.

[0006] In one optional implementation, the step of evaluating the road coupling risk index value for each risk calculation unit based on the environmental perception results and inertial measurement data includes: extracting features from the inertial measurement data and the environmental perception results of each risk calculation unit to obtain a risk assessment feature set; adjusting the feature weights corresponding to each feature vector in the risk assessment feature set based on the inertial measurement data and the road feature information marked for the current risk calculation unit in the semantic map of the drivable area; and performing weighted fusion of the feature vectors based on the feature weights corresponding to each feature vector to obtain the road coupling risk index value corresponding to the current risk calculation unit.

[0007] This invention adjusts the feature weights corresponding to each feature vector based on inertial measurement data and road feature information marked for the current risk calculation unit in the semantic map of the drivable area, so as to achieve weight tilting, prioritize the capture and quantification of key risk features, quickly identify potential risks such as loss of control and skidding, trigger early warning or decision control in advance, and effectively reduce safety hazards in high-risk scenarios.

[0008] In one optional implementation, the risk assessment feature set includes at least one of a traffic state feature vector set, a road feature vector set, and a vehicle feature vector set; the step of weightedly fusing the feature vectors based on their corresponding feature weights to obtain the road coupling risk index value corresponding to the current risk calculation unit includes: calculating the road risk index value, vehicle risk index value, and / or traffic risk index value respectively for the traffic state feature vector set, road feature vector set, and / or vehicle feature vector set corresponding to the current risk calculation unit, according to the feature weights corresponding to the feature vectors in each feature vector set; and weightedly fusing the road risk index value, the vehicle risk index value, and / or the traffic risk index value to obtain the road coupling risk index value corresponding to the current risk calculation unit.

[0009] This invention extracts three feature vector sets—traffic state, road features, and vehicle features—from inertial measurement data and environmental perception results of each risk calculation unit to achieve multi-dimensional risk index assessment, thereby improving the comprehensiveness of risk index assessment. Furthermore, the weight allocation of road features is precisely adapted to the actual driving state of vehicles and actual road data, avoiding risk assessment errors caused by the disconnect between fixed weights and complex operating conditions, and further enhancing the pertinence of risk assessment.

[0010] In one optional implementation, the environmental perception result further includes obstacle depth estimation results and vehicle status information ahead. The step of inputting the fused perception data into a multimodal neural network model to obtain the model's output environmental perception result includes: inputting the fused perception data into the multimodal neural network model so that the model performs multi-level feature extraction and fusion on the spatiotemporally aligned forward-looking visual data to obtain a multi-scale visual feature map; performing upsampling and stitching operations on the multi-scale visual feature map to obtain a fused visual feature map; and based on the fused visual feature map, performing obstacle depth estimation, drivable area semantic segmentation, and vehicle status detection ahead to obtain obstacle depth estimation results, a drivable area semantic map, and vehicle status information ahead.

[0011] This invention performs multi-level feature extraction and fusion on visual data, improving the ability to express visual features and solving problems such as loss of details in single-scale feature extraction. It provides high-quality visual feature support for subsequent tasks, and performs upsampling and stitching operations on multi-scale features to effectively compensate for the loss of details caused by resolution compression during feature extraction, thus preserving the synergistic advantages of multiple scales.

[0012] In an optional implementation, the method further includes: extracting temporal features from the inertial measurement data to obtain temporal dynamic features; fusing the drivable area semantic map, obstacle depth estimation results, and vehicle state information ahead to obtain fused visual features; performing spatial dependency modeling on the fused visual features to obtain enhanced visual features; performing vehicle dynamic evolution modeling on the temporal dynamic features to generate temporal enhanced features; performing multilayer perceptron transformation on the temporal enhanced features to obtain inertial measurement risk features; performing cross-modal fusion of the fused visual features and inertial measurement risk features through a nonlinear attention mechanism to obtain cross-modal features; determining the risk level of each drivable area in the drivable area semantic map based on the cross-modal features; and providing graded warnings based on the risk level of each drivable area the vehicle is about to enter.

[0013] This invention extracts temporal features from inertial measurement data and generates temporal enhanced features by combining vehicle dynamic evolution modeling. This fully captures the temporal evolution of vehicle dynamic states, providing richer dynamic support for subsequent risk feature extraction and risk level determination. Furthermore, it integrates core visual perception information and, through spatial dependency modeling, mines the spatial relationships between various environmental elements in the fused visual features. The resulting enhanced visual features are more distinctive and robust, providing multimodal feature support for subsequent risk level prediction.

[0014] In an optional implementation, the method further includes: acquiring environmental data and vehicle status information of the current driving of the vehicle; determining a corresponding environmental correction coefficient based on the current driving environmental data; determining a corresponding vehicle status correction coefficient based on the vehicle status information; calculating a final risk index value based on the road coupling risk index value, environmental correction coefficient, and vehicle status correction coefficient corresponding to the current risk calculation unit; and updating the road coupling risk index value corresponding to the current risk calculation unit to the final risk index value.

[0015] This invention simultaneously acquires the vehicle's current driving environment data and vehicle status information, and determines the environmental correction coefficient and vehicle status correction coefficient respectively, realizing a two-dimensional risk calibration of "environment-vehicle", which greatly reduces risk assessment error and improves the accuracy and reliability of the final risk index.

[0016] In one optional implementation, the vehicle control strategy includes at least a safe speed limit. The step of generating a corresponding vehicle control strategy based on the road coupling risk index value corresponding to the target risk calculation unit into which the vehicle is about to enter includes: determining the safe speed limit corresponding to the target risk calculation unit from a pre-constructed safe speed limit map, and performing drive control on the vehicle based on the safe speed limit. The safe speed limit map is constructed through the following steps: calculating a preset risk index power of the road coupling risk index value corresponding to each risk calculation unit to obtain a first exponential product, and taking the complement value of the first exponential product; multiplying a preset base speed limit value, the complement value, and a preset compensation coefficient to obtain the safe speed limit corresponding to each risk calculation unit; and constructing a safe speed limit map based on the safe speed limits corresponding to all risk calculation units. The preset compensation coefficient is obtained by taking the minimum value of a first ratio and 1; the first ratio is the ratio of the radius of curvature corresponding to the current risk calculation unit to a preset reference radius of curvature.

[0017] This invention constructs a safe speed map for each area within a drivable zone, enabling the identification of key safety risks such as the status of vehicles ahead, the depth of obstacles, and the degree of road slippage. This solves the critical safety blind spot that traditional speed control systems cannot perceive microscopic risks. Furthermore, it can automatically adjust the safe speed limit based on the road risk level and vehicle status, eliminating the need for frequent manual adjustments by the driver and significantly reducing the operational difficulty and cognitive load of long-distance driving.

[0018] In one optional implementation, the vehicle control strategy further includes an air conditioning compressor power adjustment strategy, a torque distribution strategy, a braking coefficient preloading strategy, and an energy recovery adjustment strategy. The generation of the corresponding vehicle control strategy based on the road coupling risk index value corresponding to the target risk calculation unit the vehicle is about to enter includes: determining the target load reduction level corresponding to the target risk calculation unit based on a preset mapping relationship between different road coupling risk index value ranges and different air conditioning power reduction levels; adjusting the air conditioning compressor power based on the target load reduction level, wherein the target load reduction level is positively correlated with the road coupling risk index value range; determining the corresponding torque distribution strategy based on the road feature information and inertial measurement data marked by the target risk calculation unit; determining the braking pressure loading level corresponding to the target risk calculation unit based on a preset mapping relationship between different road coupling risk index value ranges and different braking pressure loading levels, wherein the braking pressure loading level is positively correlated with the road coupling risk index value range; and determining the energy recovery intensity corresponding to the target risk calculation unit based on a preset mapping relationship between different road coupling risk index value ranges and different energy recovery intensities, wherein the energy recovery intensity is negatively correlated with the road coupling risk index value range.

[0019] This invention uses the road coupling risk index to coordinate and precisely adjust the power and torque distribution of the air conditioning compressor, the preload of the braking coefficient, and the intensity of energy recovery. It deeply couples road risk prediction with the control of multiple vehicle systems, thereby improving the overall driving safety, handling stability, and energy utilization rationality of the vehicle in road scenarios with different risk levels.

[0020] In one optional implementation, the risk calculation unit is a risk calculation grid, which divides the current drivable area in the drivable area semantic map into multiple risk calculation units, including: determining a cone-shaped region of interest ahead based on the vehicle's driving direction, and using the region of interest as the current drivable area; adjusting the size of the risk calculation grid based on the current vehicle speed, so as to divide the current drivable area in the drivable area semantic map into multiple risk calculation grids based on the size, wherein the vehicle speed is positively correlated with the size of the grid.

[0021] This invention focuses on dividing the risk calculation grid into regions of interest, thereby focusing on the core area, avoiding invalid calculations, and improving the accuracy and efficiency of risk assessment. Furthermore, it dynamically adjusts the grid size and divides the grid based on the current vehicle speed to adapt to the actual needs of different vehicle speed conditions, thus balancing risk assessment accuracy and computational efficiency.

[0022] Secondly, the present invention provides a vehicle control device, the device comprising: a data acquisition module for acquiring inertial measurement data and forward-looking visual data of a vehicle; a data fusion module for fusing the inertial measurement data and forward-looking visual data to obtain fused perception data; an environment perception module for inputting the fused perception data into a multimodal neural network model to obtain an environment perception result output by the model, the environment perception result including at least a drivable area semantic map; a risk index assessment module for dividing the currently drivable area in the drivable area semantic map into multiple risk calculation units, and assessing a road coupling risk index value for each risk calculation unit based on the environment perception result and the inertial measurement data, wherein the road coupling risk index value is used to quantify the driving risk level of the road under the coupling effect of multiple factors; and a vehicle control module for generating a corresponding vehicle control strategy based on the road coupling risk index value corresponding to the target risk calculation unit into which the vehicle is about to enter, and controlling the vehicle's movement based on the vehicle control strategy.

[0023] Thirdly, the present invention provides a vehicle including a controller, the controller including a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the vehicle control method of the first aspect or any corresponding embodiment described above.

[0024] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the vehicle control method of the first aspect or any corresponding embodiment thereof.

[0025] This invention has the following technical advantages: The vehicle control method provided by this invention performs spatiotemporal alignment processing on the acquired inertial measurement data and forward-looking visual data to obtain spatiotemporally synchronized fused perception data, realizing the coordinated linkage of environment and vehicle data, providing high-quality data support for subsequent precise perception and control, and is adaptable to complex driving scenarios. The fused perception data is input into a multimodal neural network model to obtain the model's output environmental perception results. These results include a drivable area semantic map, which divides the current drivable area into multiple risk calculation units. Based on the environmental perception results and inertial measurement data... The system uses quantitative data to assess the road coupling risk index value for each risk calculation unit. Based on the road coupling risk index value corresponding to the target risk calculation unit that the vehicle is about to enter, a corresponding vehicle control strategy is generated to control the vehicle's movement. This enables early identification of risk areas that the vehicle is about to enter, allowing for advance prediction and control, significantly reducing driving safety hazards. Furthermore, it fully considers the coupling effects of various factors such as real-time terrain features and obstacle states to determine the risk level of the target risk unit and generate differentiated vehicle control strategies. This results in higher control accuracy and adaptability, avoiding control errors caused by the disconnect between static rules and actual complex scenarios, and improving the driving experience and safety. Attached Figure Description

[0026] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0027] Figure 1 is a schematic flowchart of a first type of vehicle control method according to an embodiment of the present invention; Figure 2 is a schematic flowchart of a second type of vehicle control method according to an embodiment of the present invention; Figure 3 is a flowchart example of generating environmental perception results according to an embodiment of the present invention; Figure 4 is a flowchart example of calculating road risk level according to an embodiment of the present invention; Figure 5 is a structural block diagram of a vehicle according to an embodiment of the present invention; Figure 6 is a structural block diagram of a vehicle control device according to an embodiment of the present invention; Figure 7 is a schematic diagram of the hardware structure of a controller according to an embodiment of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0030] According to an embodiment of the present invention, a vehicle control method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] This embodiment provides a vehicle control method that can be used in a vehicle controller. Figure 1 is a flowchart of the vehicle control method according to an embodiment of the present invention. As shown in Figure 1, the process includes the following steps: Step S101, acquiring inertial measurement data and forward vision data of the vehicle.

[0032] This invention can continuously acquire forward-looking visual data through a forward-looking camera. Taking video stream as an example, it covers the road scene in front of the vehicle, the distribution of obstacles, lane lines and the status of vehicles in front, and can simultaneously acquire data from inertial measurement units (IMUs), including but not limited to three-axis acceleration signals to detect changes in acceleration in the longitudinal, lateral and vertical directions of the vehicle, three-axis angular velocity signals to monitor changes in vehicle attitude, including roll, pitch and yaw angular velocities, and vehicle speed signals to provide accurate vehicle motion status and time series signals in combination with wheel speed sensor data. These signals provide a time reference for the spatiotemporal alignment of visual and IMU data. This is just an example.

[0033] Step S102: The inertial measurement data and forward-looking visual data are fused to obtain fused perception data.

[0034] Before fusing the data, the embodiments of the present invention can also preprocess the data. For example, for inertial measurement data, high-frequency noise can be removed by sliding window filtering to compensate for IMU zero bias and temperature drift errors, and corrected IMU motion data can be obtained. For forward vision data, image denoising and distortion correction can be performed to remove blurry, overexposed / underexposed invalid frames and retain valid vision frames and corresponding feature information. Subsequently, the two types of corrected data can be sorted separately in ascending order of timestamp. This is just an example.

[0035] This invention addresses issues such as the asynchrony between the IMU and camera local clocks by unifying the time base and accurately correcting timestamps. For example, a unified system reference clock can be introduced, and the time offsets between the IMU local clock, the camera local clock, and the reference clock can be obtained. Based on these time offsets, the timestamps of the preprocessed inertial measurement data and the forward-looking visual data can be converted to the system reference clock timestamps, eliminating the inherent offset of the local clocks. Simultaneously, based on the extrinsic parameters of the sensor installation, the inertial measurement data and the forward-looking visual data can be unified to the target spatial coordinate system. For instance, the camera coordinate system of the forward-looking camera can be selected as the target spatial coordinate system. The inertial measurement data can be converted from the IMU body coordinate system to the target spatial coordinate system through extrinsic parameters, achieving spatial alignment. Since the IMU sampling frequency is much higher than that of the forward-looking camera, interpolation and resampling can be used to unify the two types of data to the same time series, achieving a one-to-one correspondence of timestamps. This is merely an example and not a limitation. Ultimately, the spatiotemporal alignment of the inertial measurement data and the forward-looking visual data is achieved, resulting in spatiotemporally synchronized fused perception data.

[0036] Step S103: Input the fused perception data into the multimodal neural network model to obtain the environmental perception results output by the model. The environmental perception results include at least a semantic map of the drivable area.

[0037] This invention does not limit the architecture design of the multimodal neural network model. It can be set according to actual needs. Taking a neural network model containing a Convolutional Neural Network (CNN) + Transformer architecture or a shared visual backbone network + dual-stream fusion architecture as an example, in vehicle control applications, spatiotemporally aligned fused perception data can be input into the multimodal neural network model so that the model can fuse inertial measurement motion features and visual environment features through the multimodal feature fusion module, perform environmental perception inference, and output environmental perception results, including but not limited to semantic maps of drivable areas, information on obstacles ahead, road marking information, and information on the status of vehicles ahead, etc., as an example only.

[0038] Step S104: Divide the current drivable area in the drivable area semantic map into multiple risk calculation units, and evaluate the road coupling risk index value of each risk calculation unit based on environmental perception results and inertial measurement data.

[0039] Among them, the road coupling risk index is used to quantify the driving risk level of a road under the combined effects of multiple factors.

[0040] This invention can divide the current drivable area in the semantic map of drivable areas into risk calculation units. The division of risk calculation units can be achieved using a grid partitioning method, dividing the current drivable area into multiple risk calculation units of uniform size. Each risk calculation unit is assigned a unique identifier and associated with its position coordinates in the vehicle coordinate system. This is used for refined and unitized risk assessment of the drivable area, providing specific risk assessment units for subsequent risk index calculation and control strategy generation. Then, by combining environmental perception results and inertial measurement data, core coupling factors are selected, and each type of factor is assigned a corresponding weight. Quantitative evaluation is performed based on factors such as vehicle motion state, obstacle influence, road environment, and driving scenario, along with their corresponding weights, to obtain the road coupling risk index value corresponding to each risk calculation unit. This is merely an example to achieve the output of environmental perception results based on multimodal data, improving the accuracy of environmental perception results. Experimental testing has shown that risk assessment of complex scenarios can be completed within 0.1 seconds, with a warning lead time of 2-3 seconds, effectively avoiding more than 90% of potential collision risks. Even at night or in severe weather conditions, the system still maintains a risk identification accuracy of 87.4%, which is significantly better than traditional single-modality solutions.

[0041] Step S105: Based on the road coupling risk index value corresponding to the target risk calculation unit that the vehicle is about to enter, generate a corresponding vehicle control strategy, and control the vehicle's driving based on the vehicle control strategy.

[0042] This invention can predict the target risk unit that the vehicle is about to enter based on the vehicle's current speed, steering angle, and the location coordinates of the risk calculation unit. Then, based on the road coupling risk index value corresponding to the target risk calculation unit and the vehicle's current speed, a differentiated vehicle control strategy is generated. For example, if the road coupling risk index is low, the current driving state can be maintained; if the road coupling risk index is medium, the vehicle can decelerate, adjust the steering angle, and turn on the warning lights; if the road coupling risk index is high, the vehicle can decelerate urgently to a safe speed and turn off the acceleration function. This is just an example; the vehicle's driving can be controlled based on the corresponding vehicle control strategy in the future.

[0043] The vehicle control method provided by this invention fuses acquired inertial measurement data and forward-looking visual data to obtain fused perception data, achieving coordinated linkage between environmental and vehicle data. This provides high-quality data support for subsequent precise perception and control, and is adaptable to complex driving scenarios. The fused perception data is input into a multimodal neural network model to obtain the environmental perception results output by the model. The environmental perception results include a drivable area semantic map, which divides the currently drivable area into multiple risk calculation units. Based on the environmental perception results and inertial measurement data, the road coupling risk index value is evaluated for each risk calculation unit. Based on the road coupling risk index value corresponding to the target risk calculation unit that the vehicle is about to enter, a corresponding vehicle control strategy is generated to control the vehicle's movement. This method enables early identification of risk areas that the vehicle is about to enter, allowing for advance prediction and control, significantly reducing driving safety hazards. Furthermore, it fully considers the coupling effects of various factors such as real-time terrain features and obstacle states to determine the risk level of the target risk unit and generate differentiated vehicle control strategies. This results in higher control accuracy and adaptability, avoiding control errors caused by the disconnect between static rules and actual complex scenarios, and improving driving experience and safety.

[0044] This embodiment provides a vehicle control method that can be used in a vehicle controller. Figure 2 is a flowchart of the vehicle control method according to an embodiment of the present invention. As shown in Figure 2, the process includes the following steps: Step S201, acquiring inertial measurement data and forward vision data of the vehicle. For details, please refer to step S101 of the embodiment shown in Figure 1, which will not be repeated here.

[0045] Step S202: The inertial measurement data and forward-looking visual data are fused to obtain fused perception data.

[0046] Specifically, the two types of data can be fused through the following steps: Nonlinear matching is performed between the video frame time series in the forward-looking visual data and the sampling time series of the inertial measurement data to obtain the inertial measurement data associated with the timestamp of each video frame; Based on sensor calibration parameters and motion compensation algorithms, the vehicle motion state reflected by the inertial measurement data is spatially mapped with the environmental features reflected by the forward-looking visual data; Based on the video frame sequence, the visual displacement corresponding to the first moment is calculated, and the inertial measurement data is integrated to obtain the estimated displacement corresponding to the timestamp matched with the first moment; When the difference between the visual displacement and the estimated displacement exceeds a preset tolerance range, the visual data of the first moment and the inertial measurement data corresponding to the matched timestamp are determined to be abnormal data; Abnormal data is removed to obtain spatiotemporally synchronized fused perception data.

[0047] The embodiments of the present invention may employ dynamic time warping. The DTW (Time-Digital Warping) method performs non-linear matching between the time series of video frames in forward-looking visual data and the sampling time series of inertial measurement data. By calculating the similarity between the sequences, it finds the optimal matching path and ultimately associates each video frame with the inertial measurement data whose timestamp best matches, ensuring strict time synchronization between the forward-looking visual data and the inertial measurement data and eliminating data mismatch caused by time offset. Simultaneously, based on sensor calibration parameters and motion compensation algorithms, it spatially maps the vehicle motion state reflected by the inertial measurement data and the environmental features reflected by the forward-looking visual data. Specifically, it can call sensor calibration parameters (such as rotational torque and translation vector) to initially convert the IMU-corrected data from the IMU body coordinate system to the visual camera coordinate system. Considering the vibration and attitude changes that occur during vehicle movement, a motion compensation algorithm can be used to perform spatial attitude compensation on the visually perceived environmental features based on the vehicle attitude parameters (such as angular velocity and angular acceleration) measured in real time by the IMU, correcting the spatial offset of environmental features caused by vehicle motion. Assisted positioning using IMU data achieves accurate spatial correspondence, eliminates visual jitter, and improves the accuracy of environmental perception results. This is just one example.

[0048] This invention can calculate the visual displacement corresponding to the first moment (the moment corresponding to each video frame) based on a video frame sequence, reflecting the actual displacement change of the vehicle relative to the environment. By performing a second integration on the acceleration data in the inertial measurement data, an estimated displacement corresponding to the timestamp matching the first moment is obtained, reflecting the displacement change derived from the vehicle's dynamic parameters. Then, the visual displacement and the estimated displacement can be input into a Kalman filter. Through the filter's state estimation and error correction functions, the displacement error between the two is calculated. Then, the displacement error is compared with a preset tolerance range (which can be set to ±0.05m). When the displacement error exceeds this tolerance range, the visual displacement at the first moment is determined. Inertial measurement data corresponding to the data and matching timestamps are considered abnormal data. Abnormal data is marked and removed to accurately identify abnormal data caused by sensor errors and external interference. By removing abnormal data, the effectiveness and accuracy of the fused perception data are ensured, resulting in fused perception data that has undergone time alignment, spatial alignment, and consistency verification. This ensures three core correspondences: the visually recognized obstacle position corresponds precisely in time and space to the vehicle motion state perceived by the IMU; the visual signal of the taillight flashing is strictly synchronized with the braking acceleration change of the vehicle in front detected by the IMU; and the drivable area segmentation result is accurately correlated with the vehicle's real-time attitude parameters.

[0049] This invention achieves deep and accurate fusion of forward-looking visual data and inertial measurement data by performing time and space alignment and consistency verification on the two types of data. This completely solves the problem that the two types of core data are independent and not effectively utilized, and provides high-quality and reliable data support for subsequent risk assessment and vehicle control.

[0050] Step S203: Input the fused perception data into the multimodal neural network model to obtain the environmental perception results output by the model.

[0051] The environmental perception results include at least a semantic map of the drivable area, obstacle depth estimation results, and vehicle status information ahead. The environmental perception results can be output through a multimodal neural network model via the following steps: The fused perception data is input into the multimodal neural network model, enabling the model to perform multi-level feature extraction and fusion on the spatiotemporally aligned forward-looking visual data to obtain a multi-scale visual feature map; the multi-scale visual feature map is upsampled and stitched to obtain a fused visual feature map; based on the fused visual feature map, obstacle depth estimation, drivable area semantic segmentation, and vehicle status detection ahead are performed to obtain obstacle depth estimation results, a drivable area semantic map, and vehicle status information ahead.

[0052] In this embodiment of the invention, fused perceptual data can be input into a multimodal neural network model. The spatiotemporally aligned forward-looking visual data undergoes convolution and two layers of CBS (Convolution-BatchNorm-SiLU, a lightweight feature extraction module) and C2f (Cross Stage Partial connections with 2 convolutions and feature fusion, an efficient feature fusion and enhancement module) to obtain a basic visual feature map (feature1). This basic visual feature map retains rich low-level visual features and has a spatial resolution of 320×180. After further processing with two layers of CBS and C2f, the basic visual feature map yields a mid-level semantic feature map (feature2), which extracts mid-level semantic information and has a spatial resolution of 160×90. After further processing with two layers of CBS and C2f, the mid-level semantic feature map yields a high-level semantic feature map (feature3), which contains high-level semantic features and has a spatial resolution of 80×45. This is just an example; the process architecture is shown in Figure 3.

[0053] In this embodiment of the invention, a bottom-up visual feature pyramid structure can be designed. By fusing multi-scale features of the above three features, hierarchical feature extraction from local details to global semantics is achieved. This visual feature pyramid fusion mechanism, along with upsampling and stitching operations, deeply fuses features at different scales. While maintaining a high frame rate inference speed, the depth estimation accuracy reaches 93.8%, and the IoU of drivable region segmentation is improved to 92.0%, which is significantly better than single-scale feature extraction methods. Specifically, the feature pyramid fusion mechanism can perform upsampling and stitching operations on the basic visual feature map, intermediate semantic feature map, and high-level semantic feature map to obtain the fused visual feature map feature4. This fused feature map combines multi-scale information and has a spatial resolution of 320×180, which enhances semantic understanding capabilities while preserving spatial details.

[0054] This invention utilizes a squeeze-and-excitation attention (SE-Attention) mechanism to process the fused visual feature map through global average pooling and learning channel weights in a fully connected layer. This process highlights important feature channels and suppresses irrelevant channels, resulting in an output single-channel depth map (feature 5) for subsequent obstacle depth estimation tasks. The fused visual feature map is then convolved to obtain a probability map (feature 6) of the drivable region, used for semantic segmentation tasks—drivable region segmentation—with a resolution of 640×360. The high-level semantic feature map (feature 3) is processed using fast spatial pyramid pooling. The Fast (SPPF) module and convolutional operations yield a comprehensive visual feature map. The SPPF module enhances the receptive field through max pooling operations at different scales. Feature7 is used for object detection tasks—obstacle and vehicle taillight detection, outputting bounding box coordinates, class probabilities, and confidence scores. Finally, the obstacle depth estimation results, drivable area semantic map, and vehicle status information ahead are obtained. As an example, the drivable area semantic map can include pixel-level segmentation results of 15 drivable areas, including key information such as lanes, shoulders, emergency lanes, construction areas, and waterlogged areas. The obstacle depth estimation results provide three-dimensional spatial distribution information of obstacles ahead, including distance, size, and relative position. The vehicle status information ahead includes, but is not limited to, taillight status (brake lights, turn signals, hazard lights) and vehicle motion status (acceleration changes, steering intention).

[0055] This invention performs multi-level feature extraction and fusion on visual data, improving the ability to express visual features and solving problems such as loss of details in single-scale feature extraction. It provides high-quality visual feature support for subsequent tasks, and performs upsampling and stitching operations on multi-scale features to effectively compensate for the loss of details caused by resolution compression during feature extraction, thus preserving the synergistic advantages of multiple scales.

[0056] In one optional implementation, this invention designs a multi-task dynamic weighted loss function, which can automatically adjust the loss weights according to the training difficulty and convergence speed of each task. The depth estimation task uses a loss function, the semantic segmentation task uses cross-entropy loss to extract the state of vehicles ahead, and the risk assessment task uses KL divergence loss. The invention employs a weighted uncertainty mechanism to adaptively adjust the learning rate of each task. Furthermore, to enhance the model's generalization ability and robustness, a cross-modal knowledge distillation mechanism is introduced. During the training phase, a large teacher model (12.5M parameters) guides the learning of a lightweight student model (4.2M parameters). The teacher model acquires rich environmental understanding through multimodal pre-training, while the student model learns cross-modal semantic associations by mimicking the intermediate feature distribution and final output of the teacher model. This mechanism keeps the performance degradation of the student model under adverse conditions such as rain, fog, and nighttime within 8%, significantly improving the model's environmental adaptability. The invention also adopts a progressive training strategy of "visual priority, gradual fusion." In the early stages of training, the visual encoder, multi-task decoder, and fixed IMU feature extractor are primarily optimized. In the middle stages of training, IMU features are gradually introduced, and the weights of the fusion module are adjusted. In the later stages of training, the entire multimodal fusion architecture is comprehensively optimized. This progressive training strategy effectively avoids gradient conflicts in the early stages of training, making model convergence more stable.

[0057] In an optional implementation, the present invention may further extract temporal features from inertial measurement data to obtain temporal dynamic features; fuse the drivable area semantic map, obstacle depth estimation results, and vehicle state information ahead to obtain fused visual features; perform spatial dependency modeling on the fused visual features to obtain enhanced visual features; perform vehicle dynamic evolution modeling on the temporal dynamic features to generate temporal enhanced features; perform multilayer perceptron transformation on the temporal enhanced features to obtain inertial measurement risk features; perform cross-modal fusion of the fused visual features and inertial measurement risk features through a nonlinear attention mechanism to obtain cross-modal features; determine the risk level of each drivable area in the drivable area semantic map based on the cross-modal features; and perform graded warnings based on the risk level of each drivable area that the vehicle is about to enter.

[0058] This invention embodiment can simultaneously perform 5 layers of dense connections, batch normalization (BN), rectified linear activation function (ReLU), and dropout operations on inertial measurement data (including but not limited to triaxial acceleration, triaxial angular velocity, vehicle speed, and time series) to obtain temporal dynamic features (feature 8) containing vehicle dynamic states. This invention embodiment can also simultaneously input the drivable area semantic map, obstacle depth estimation results, and vehicle state information ahead into a 3×3 convolution operation to obtain fused visual features (feature 9). This convolutional layer unifies the multi-task output into a 256-dimensional feature vector with a spatial resolution of 40×22, containing rich environmental semantic information. Then, the fused visual features are processed through a self-attention mechanism to obtain enhanced visual features (feature 10). The self-attention mechanism models long-distance spatial dependencies by calculating the correlation between feature positions. In specific applications, multi-head self-attention can be used. The Self-Attention structure divides the features into 8 heads for parallel computation and then merges the results, enabling the model to simultaneously focus on risk features in different regions, significantly improving its ability to understand complex scenarios. As an example, this invention clearly depicts the spatial distribution features of the driving environment by fusing visual data such as semantic maps of drivable areas, obstacle depth, and the status of the vehicle in front and modeling spatial dependencies.

[0059] In this embodiment of the invention, temporal dynamic feature feature8 is processed through a Long Short-Term Memory (LSTM) network and convolutional operations to obtain temporal enhanced feature feature11. The LSTM network models the temporal signal of the IMU, capturing the temporal evolution of the vehicle's dynamic state. The LSTM hidden layer has a dimension of 128, effectively memorizing historical motion states. By extracting temporal features from inertial measurement data and modeling vehicle dynamic evolution separately, this invention can accurately capture the dynamic changes of the vehicle itself, achieving full coverage of both vehicle dynamics and external environmental spatial features, avoiding information omissions caused by single feature extraction. The subsequent 1×1 convolutional layer adjusts the feature dimension to 256, aligning it with the visual feature dimension. Then, the temporal enhanced feature feature11 can be processed. Re11 undergoes a Multi-Layer Perceptron (MLP) transformation to obtain the inertial measurement risk feature12. The MLP contains two fully connected layers (256→128→64). Through nonlinear transformation, a high-order abstract representation of the IMU features is extracted to remove redundant raw inertial data information and focus on the vehicle's motion trend, stability state, and potential risk information. The dimensions are also gradually compressed to reduce the data dimensionality while retaining the core features, thereby reducing the computational load of subsequent cross-modal fusion and risk calculation and improving the system's processing efficiency. At the same time, the output dimension and representation form of the obtained standardized inertial measurement risk features can be efficiently matched with the fused visual features, laying the foundation for subsequent cross-modal fusion using nonlinear attention mechanisms and ensuring the consistency and reliability of the fused features.

[0060] This invention embodiment utilizes a non-linear attention mechanism to perform cross-modal fusion of the fused visual feature (feature 10) and the inertial measurement risk feature (feature 12), resulting in a cross-modal feature (feature 13). The core of this non-linear attention mechanism is calculating the non-linear correlation weights between the visual feature and the IMU feature: First, feature 10 and feature 12 are mapped to the same feature space through a linear transformation. Then, the non-linear correlation is calculated: W = tanh(θ·[feature 10, feature 12] + ... (b) where θ and b are learnable parameters. Finally, attention weights are obtained through Softmax normalization, and the features are weighted and fused. The nonlinear attention mechanism can adaptively adjust the contribution ratio of visual features and IMU features. For example, it increases the weight of lateral acceleration features in curved scenes and increases the weight of visual semantic features in slippery road scenes. It can also adaptively learn the complex nonlinear relationship between visual semantic features and IMU temporal features. Compared with traditional attention mechanisms, this invention significantly improves the expressive power of cross-modal feature fusion by introducing a nonlinear transformation function, improving the risk assessment accuracy by 15.2%. Furthermore, this invention performs spatial modeling of visual semantic features through a self-attention mechanism, performs temporal modeling of IMU signals through an LSTM network, and finally achieves deep fusion of spatiotemporal features through a nonlinear attention mechanism. This spatiotemporal joint modeling architecture can simultaneously capture the spatial distribution characteristics of road scenes and the temporal evolution of vehicle dynamics. It is particularly suitable for assessing the risk level of complex scenarios such as continuous curves and slope changes. It can automatically focus on key risk-related features (such as the correlation between obstacle depth and vehicle acceleration, and the correlation between the braking state of the vehicle in front and the deceleration trend of the vehicle itself), achieving deep collaboration of "environment-vehicle" multimodal features. This provides high-quality cross-modal feature support for risk level determination. Furthermore, it addresses the asynchronous problem between visual data and IMU signals in the spatiotemporal dimension by modeling the temporal signals of the IMU through an LSTM network, extracting vehicle dynamic state features, and then deeply fusing them with visual semantic features through a nonlinear attention mechanism. This achieves accurate alignment between visual spatial information and IMU temporal information, reducing the cross-modal feature matching error to within 5.1ms.

[0061] This invention allows cross-modal feature13 to be processed through a linear layer, dropout (with a ratio of 0.3), and activation function operations to obtain risk classification features, which are then used for classifying road risk levels. The linear layer maps the features to a 5-dimensional output space, corresponding to multiple risk levels (taking 5 risk levels: extremely low, low, medium, high, and extremely high as an example). The output is converted into a probability distribution using a softmax function, and the level with the highest probability is selected as the risk assessment result. For example, the risk classification features represent the risk level of each drivable area. The process architecture is shown in Figure 4. This invention determines the risk level of each area within a drivable region based on cross-modal features, achieving unitized and refined risk assessment of drivable areas. This avoids the omission of local high-risk areas caused by holistic assessment, significantly improving the accuracy and targeting of risk identification.

[0062] This invention can provide tiered warnings based on the risk level of each drivable area the vehicle is about to enter. For example, if the risk level is extremely low, the current driving state can be maintained without warning. If the risk level is low, a visual display (yellow warning) can be displayed on the dashboard. If the risk level is medium, a visual and audible warning (red warning) can be issued. If the risk level is high, a warning can be issued through visual, audible, and seat vibration (red warning). The warning lead time can reach 2.5-3.5 seconds, which is only an example.

[0063] This invention extracts temporal features from inertial measurement data and generates temporal enhanced features by combining vehicle dynamic evolution modeling. This fully captures the temporal evolution of vehicle dynamic states, providing richer dynamic support for subsequent risk feature extraction and risk level determination. Furthermore, it integrates core visual perception information and, through spatial dependency modeling, mines the spatial relationships between various environmental elements in the fused visual features. The resulting enhanced visual features are more distinctive and robust, providing multimodal feature support for subsequent risk level prediction.

[0064] Step S204: Divide the current drivable area in the drivable area semantic map into multiple risk calculation units, and evaluate the road coupling risk index value of each risk calculation unit based on environmental perception results and inertial measurement data.

[0065] Among them, the road coupling risk index is used to quantify the driving risk level of a road under the combined effects of multiple factors.

[0066] Specifically, the risk calculation unit designed in this invention is a risk calculation grid. The above step S204 includes: step S2041, determining the region of interest of the cone ahead according to the vehicle's driving direction, and taking the region of interest as the current drivable area.

[0067] Step S2042: Adjust the size of the risk calculation grid according to the current vehicle speed, so as to divide the current drivable area in the drivable area semantic map into multiple risk calculation grids based on the size. Among them, the vehicle speed is positively correlated with the grid size.

[0068] This invention can acquire the semantic map of the drivable area, IMU inertial measurement data (to extract the vehicle's current driving direction and accurately reflect the vehicle's straight-ahead, turning, and other postures), and the vehicle's current speed determined in the above steps. Then, combined with the perception range of the vehicle's forward-facing camera, core parameters of the cone-shaped region of interest can be set. For example, cone positioning can be defined as the coordinates of the vehicle's front center point in the semantic map of the drivable area. The cone's base is located at the perception limit in front of the vehicle (e.g., 100m). The width of the base is adapted to the current lane width and the vehicle's driving direction. In straight-ahead conditions, the base width can be equal to 1.2 times the current lane width to reserve safety redundancy. In turning conditions, the base width can adaptively widen according to the turning radius; the smaller the turning radius, the greater the widening, ensuring coverage of the entire area the vehicle will enter during the turn. The drivable area, used as an example only, is designed to ensure comprehensive and redundant risk coverage due to the adaptive characteristics of the cone shape. The cone-shaped region of interest (ROI) adopts a "narrower at the front and wider at the back" structure, conforming to the vehicle's field of vision and potential driving trajectory. The apex corresponds to the front of the vehicle, and the bottom extends to the frontal perception limit. This comprehensively covers all drivable areas the vehicle may encounter during future driving, avoiding omissions of risk areas caused by turning or lane changes. It also avoids computational redundancy caused by excessively widening the area. Furthermore, it can dynamically update in real-time following the vehicle's driving direction and front position, ensuring the ROI is always synchronized with the vehicle's driving status. This adapts to different driving conditions such as straight-line driving and turning, solving the pain point that traditional fixed-range divisions cannot adapt to dynamic vehicle movement. Alternatively, the ROI can be directly determined using the following formula, and then used as the drivable area:

[0069] Where ROI represents the region of interest, and x and y are the center coordinates of the grid.

[0070] This invention embodiment can adjust the size of the risk calculation grid according to the current vehicle speed. The grid resolution can be dynamically adjusted according to the vehicle speed using the following formula: grid_size=max(3,min(10,0.05) vc+3)) where grid_size represents the size of the risk calculation grid; vc represents the current vehicle speed. For example, a 7×7 meter grid can be used at high speeds and a 3×3 meter grid can be used at low speeds.

[0071] In a specific embodiment, the drivable area of ​​200×200 meters in front of the vehicle can be divided into a dynamic grid of 5×5 meters. The current vehicle position is set as the origin (0,0), and the direction of the vehicle's front is the positive X-axis, establishing a world coordinate system. The position of each grid is determined by its center coordinates (xi, yj), where: xi = i×5 + 2.5, i = 0, 1, 2, ..., 39, yj = j×5 + 2.5, j = ... 19, 18,...,19,20, with a total grid size N=40×40=1600, the image coordinates can be mapped to world coordinates using a perspective transformation matrix.

[0072] Where (u,v) are the image pixel coordinates, , () represents world coordinates.

[0073] Furthermore, in the subsequent calculation of the road coupling risk index value corresponding to each risk calculation grid, this embodiment of the invention can utilize the Single Instruction Multiple Data (SIMD) architecture of the GPU to parallelize the extraction of multi-dimensional features (taking 11-dimensional features as an example in this embodiment). This reduces the computational complexity for N risks from O(N0) to O(N0). 11) Optimize to O(N) 11 / P), where O represents the mathematical symbol for computational complexity; P is the number of parallel processing units (usually P=256) to reduce risk computational complexity.

[0074] In one alternative implementation, when the vehicle travels less than half the grid size, only the affected grid can be updated, achieving an update ratio typically less than 30%, which significantly reduces the computational load.

[0075] This invention focuses on dividing the risk calculation grid within the region of interest, achieving the effect of focusing on the core area, avoiding invalid calculations, and improving the accuracy and efficiency of risk assessment. Furthermore, it dynamically adjusts the grid size and division based on the current vehicle speed to adapt to the actual needs of different vehicle speed conditions, balancing risk assessment accuracy and computational efficiency. Specifically, the road coupling risk index value corresponding to each risk calculation unit can be calculated through the following steps: Feature extraction is performed on inertial measurement data and the environmental perception results of each risk calculation unit to obtain a risk assessment feature set; based on the inertial measurement data and the road feature information marked for the current risk calculation unit in the semantic map of the drivable area, the feature weights corresponding to each feature vector in the risk assessment feature set are adjusted; based on the feature weights corresponding to each feature vector, the feature vectors are weighted and fused to obtain the road coupling risk index value corresponding to the current risk calculation unit.

[0076] This invention considers that environmental perception results, which characterize road environment information, do not include IMU dynamics features. These results are susceptible to environmental influences such as occlusion, rain, and fog, leading to data jumps, delays, and partial missing values. In contrast, the inertial measurement unit (IMU) can output high-frequency, continuous, and environmentally resistant vehicle attitude, acceleration, and angular velocity data. Therefore, by combining IMU inertial measurement data with environmental perception results from various risk calculation units in calculating the road coupling risk index, a comprehensive assessment of environmental risk and vehicle state information is obtained, resulting in a more accurate and realistic drivable risk assessment. This improves the robustness of risk assessment, overcomes the failure of single perception units, and provides a foundation for autonomous driving systems. This provides more accurate and reliable technical support for risk warning and decision control; therefore, features can be extracted from inertial measurement data (such as vehicle attitude, acceleration, angular acceleration, friction coefficient, etc., reflecting the vehicle's own motion state) and environmental perception results of each risk calculation unit (such as road surface conditions: slipperiness, radius of curvature, etc., and surrounding traffic participant conditions: distance to the vehicle ahead, vehicle density, etc., reflecting the road environment). Core features related to road coupled risks can be screened out and integrated to form a risk assessment feature set. Then, based on the collected inertial measurement data and the road features marked for the current risk calculation unit in the semantic map of the drivable area, the system can be used to assess the risks. The system dynamically adjusts the feature weights corresponding to each feature vector in the risk assessment feature set based on the information collected. This adjustment process aligns with the actual scenario requirements. For example, if the semantic map marks the current risk calculation unit as a waterlogged area, the weight of the slipperiness feature is automatically increased; if it's marked as an icy or snowy road surface, the weight of the friction coefficient feature is significantly increased. In a curve scenario, the weights can be set to 0.35 for the radius of curvature, 0.25 for lateral acceleration, and 0.20 for the friction coefficient, with the remaining feature weights set to 0. This is just an example to ensure that the weight allocation focuses on the core risk points of the current scenario, avoiding the drawbacks of fixed weights failing to adapt to the differences in the current scenario, and ensuring that the road coupling... The combined risk index can accurately reflect the actual level of danger during driving, significantly reducing assessment bias. After adjusting the weights of each feature vector, the weighted fusion calculation can be performed on each feature vector based on the adjusted feature weights, converting multi-dimensional feature information into quantified values, thus obtaining the road coupled risk index value corresponding to the current risk calculation unit. This enables accurate risk index-based assessment and risk identification, allowing for early warning mechanisms or proactive decision-making and control, effectively avoiding safety hazards in high-risk scenarios. For example, in scenarios prone to accidents such as ice and snow or water accumulation, it can significantly reduce the probability of accidents and improve vehicle driving safety.

[0077] Furthermore, the risk assessment feature set includes at least one of the traffic state feature vector set, road feature vector set, and vehicle feature vector set. The following steps, based on the feature weights corresponding to each feature vector, are used to weight and fuse the feature vectors to obtain the road coupling risk index value corresponding to the current risk calculation unit: For the traffic state feature vector set, road feature vector set, and / or vehicle feature vector set corresponding to the current risk calculation unit, calculate the road risk index value, vehicle risk index value, and / or traffic risk index value respectively according to the feature weights corresponding to the feature vectors in each feature vector set; and weight and fuse the road risk index value, vehicle risk index value, and / or traffic risk index value to obtain the road coupling risk index value corresponding to the current risk calculation unit.

[0078] The environmental perception results calculated in this embodiment of the invention also include obstacle depth estimation results and vehicle status information ahead. Considering that road coupling risks are affected by multiple dimensions of risks, such as road risk (representing driving constraints and loss of control risks at the road infrastructure level), vehicle risk (representing stability and handling risks at the vehicle's own motion level), and traffic risk (representing congestion and conflict risks at the traffic flow level), the invention quantifies dynamic traffic risks such as rear-end collisions and cutting in. These three risks have different sources, mechanisms, and weights. Therefore, vehicle feature vector sets corresponding to each risk calculation unit can be extracted based on inertial measurement data, and traffic state feature vector sets and road feature vector sets corresponding to each risk calculation unit can be extracted based on the environmental perception results. The road feature vector set includes, but is not limited to, the friction coefficient. Slipperyness radius of curvature Slope angle Traffic state feature vector sets include, but are not limited to, the density of vehicles ahead. closest distance Headlight status The vehicle feature vector set includes, but is not limited to, the current vehicle speed. Load (m), lateral acceleration coefficient of friction As an example, this invention decomposes complex road coupling risks into three independent dimensions: road, vehicle, and traffic, corresponding to three types of feature vector sets. This clarifies the sources and mechanisms of risk in different dimensions, solving the shortcomings of traditional holistic risk assessments that cannot locate the source of risk or explain its causes. This makes the risk assessment results more targeted and facilitates precise subsequent control. Furthermore, all three types of feature vector sets contain specific features that fit actual driving scenarios, comprehensively covering the core risk factors of road infrastructure, vehicle movement, and traffic flow environment. This avoids the problem of missing key risk points in a single feature dimension, making the risk feature representation more comprehensive and refined. It is no longer limited to static road structures but extends to dynamic traffic scenarios, significantly improving the safety and robustness of autonomous driving decisions. This provides accurate data support for subsequent risk index calculations and is linked to the road feature vector weight adjustment described below, realizing a complete closed loop of feature extraction—weight adjustment—index calculation, making the entire risk assessment system more logical and complete.

[0079] This invention can adjust the feature weights corresponding to each feature vector in the road feature vector set based on inertial measurement data and road feature information marked for the current risk calculation unit in the semantic map of drivable areas. For example, if the risk calculation unit is marked as a waterlogged area in the semantic map, the slipperiness feature will be automatically assigned a higher weight. When marked as an icy or snowy road surface, the friction coefficient feature weight coefficient will be significantly increased. In a curve scenario, the radius of curvature weight = 0.35, the lateral acceleration weight = 0.25, the friction coefficient weight = 0.20, and other feature weights = 0. The method of adjusting the feature weights corresponding to each feature vector in the road feature vector set is illustrated as an example. The method for adjusting the feature weights corresponding to each feature vector in the vehicle feature vector set can also be adjusted accordingly based on the actual traffic conditions and vehicle operating status. This will not be elaborated here. The goal is to enable the road feature vector set to accurately match the core risks of the current driving scenario, avoid risk assessment bias caused by fixed weights, make the road coupled risk index more in line with the actual driving danger level, and for scenarios prone to danger, the system can prioritize capturing and quantifying key risk features through weight tilting, quickly identify potential risks such as loss of control and skidding, trigger early warnings or decision control, effectively reduce safety hazards in high-risk scenarios, and make up for the shortcomings of traditional fixed weight assessment in adapting to scenario differences.

[0080] The road risk index value can be calculated using the following formula in embodiments of the present invention:

[0081] in, This indicates the road risk index value; This represents the weighting coefficient corresponding to the radius of curvature; This represents the weighting coefficient corresponding to the slope angle; This represents the weighting coefficient corresponding to the friction coefficient; This represents the weighting coefficient corresponding to the degree of slipperiness. Represents the normalization function:

[0082] Where k is the slope parameter, x0 is the threshold corresponding to different features, and x represents the physical quantity corresponding to different road features.

[0083] The vehicle risk index value can be calculated using the following formula in embodiments of the present invention:

[0084] in, This indicates the vehicle risk index value; This indicates the minimum turning radius.

[0085] The traffic risk index value can be calculated using the following formula in embodiments of the present invention:

[0086] in, λ represents the traffic risk index value; λ represents the adjustment parameter.

[0087] This invention employs a weighted fusion of road risk index, vehicle risk index, and traffic risk index to obtain the road coupling risk index corresponding to the current risk calculation unit, as shown in the following formula:

[0088] in, This represents the road coupling risk index value corresponding to the current risk calculation unit; , , These are weighting coefficients, which can be dynamically adjusted according to the driving mode.

[0089] This invention extracts three feature vector sets—traffic state, road features, and vehicle features—from inertial measurement data and environmental perception results of each risk calculation unit to achieve multi-dimensional risk index assessment, thereby improving the comprehensiveness of risk index assessment. Furthermore, the weight allocation of road features is precisely adapted to the actual driving state of vehicles and actual road data, avoiding risk assessment errors caused by the disconnect between fixed weights and complex operating conditions, and further enhancing the pertinence of risk assessment.

[0090] Furthermore, it can also acquire the environmental data and vehicle status information of the vehicle's current driving; determine the corresponding environmental correction coefficient based on the current driving environmental data; determine the corresponding vehicle status correction coefficient based on the vehicle status information; calculate the final risk index value based on the road coupling risk index value, environmental correction coefficient, and vehicle status correction coefficient corresponding to the current risk calculation unit; and update the road coupling risk index value corresponding to the current risk calculation unit to the final risk index value.

[0091] After calculating the road coupling risk index value corresponding to each risk calculation unit, this embodiment of the invention can obtain the current driving environment data and vehicle status information. The current driving environment data mainly includes weather conditions (sunny, rainy, foggy, etc.), light intensity, road surface adhesion coefficient, visibility, etc., while the vehicle status information includes at least load, tire wear, and battery temperature. Then, based on the obtained current driving environment and combined with preset coefficient calibration rules, the corresponding environmental correction coefficient can be determined. This environmental correction coefficient is a value greater than 0, used to quantify the degree of influence of environmental factors on risk. For example, the environmental correction coefficient is 1.2 in rainy / foggy weather, 1.15 at night, and 1.05 in strong light. Based on the acquired vehicle current status information and combined with preset coefficient calibration rules, the corresponding vehicle status correction coefficient can be determined. This vehicle status correction coefficient is also a value greater than 0, used to quantify the impact of the vehicle's own status on risk. For example, when the load is >80%, the vehicle status correction coefficient is 1.1; when tire wear is >50%, the vehicle status correction coefficient is 1.08; and when the battery temperature is >45°C, the vehicle status correction coefficient is 1.05 (this is just an example). Then, the road coupling risk index value corresponding to the current risk calculation unit can be multiplied by the environmental correction coefficient and the vehicle status correction coefficient to obtain the final risk index value. Finally, the road coupling risk index value corresponding to the current risk calculation unit is updated to the final risk index value (this is just an example).

[0092] This invention simultaneously acquires the vehicle's current driving environment data and vehicle status information, and determines the environmental correction coefficient and vehicle status correction coefficient respectively, realizing a two-dimensional risk calibration of "environment-vehicle", which greatly reduces risk assessment error and improves the accuracy and reliability of the final risk index.

[0093] In one optional implementation, the road coupling risk index value of the current risk calculation unit at the current moment, as well as the road coupling risk index values ​​of multiple adjacent risk calculation units in space, can also be obtained. Based on all feature vectors extracted from the current risk calculation unit and all feature vectors extracted from each adjacent risk calculation unit, spatial attention weights are calculated. Based on the time difference between the current risk calculation unit at the current moment and historical moments, time decay attention weights corresponding to each moment are calculated. The road coupling risk index values ​​of adjacent risk calculation units are spatially weighted and coupled according to the spatial attention weights, and the risk indices of historical moments are temporally weighted and fused according to the time decay attention weights to obtain the optimized risk index of the current risk calculation unit after spatiotemporal fusion.

[0094] Specifically, for the current risk calculation unit and its 8 neighboring units, the spatial attention weight can be calculated using the following formula:

[0095] in, The spatial attention weight of a certain neighboring grid to the current risk calculation unit; This refers to the 8-neighborhood of the current risk calculation unit; Cosine similarity; The multidimensional risk feature vectors representing neighborhood units (i.e., road feature vector sets, vehicle feature vector sets, and traffic feature vector sets); This is the multidimensional risk feature vector of the current risk calculation unit.

[0096] The embodiments of the present invention take into account historical risk status. The time-decay attention weight can be calculated using the following formula:

[0097] in, The influence weight of historical moments on the current moment (time decay attention weight). This represents the risk index of the current risk calculation unit at historical time t-1; This represents the time decay coefficient (taken as 0.3).

[0098] The embodiments of the present invention can calculate the optimized risk index of the current risk calculation unit after spatiotemporal fusion using the following formula:

[0099] in, This represents the optimized risk index value of the current risk calculation unit after spatiotemporal fusion; This represents the risk index value after spatial fusion; This represents the risk index value after time fusion.

[0100] This invention integrates spatial and temporal contexts on the basis of the basic road coupling risk index value. Spatial integration reduces the risk difference between adjacent grids by 42%, and temporal integration reduces the standard deviation of risk fluctuation by 68%, ultimately improving vehicle speed control smoothness by 35% and reducing unnecessary powertrain adjustments.

[0101] Step S205: Based on the road coupling risk index value of adjacent risk calculation units according to the spatial attention weight, a corresponding vehicle control strategy is generated based on the road coupling risk index value corresponding to the target risk calculation unit into which the vehicle is about to enter, and the vehicle driving is controlled based on the vehicle control strategy.

[0102] Specifically, step S205 includes: step S2051, determining the safe speed limit corresponding to the target risk calculation unit from the pre-constructed safe speed limit map, so as to perform drive control on the vehicle based on the safe speed limit.

[0103] The safe speed limit map is constructed through the following steps: Calculate the preset risk index power of the road coupling risk index value corresponding to each risk calculation unit to obtain the first exponential product, and take the complement value of the first exponential product; multiply the preset basic speed limit value, the complement value, and the preset compensation coefficient to obtain the safe speed limit value corresponding to each risk calculation unit; and construct the safe speed limit map based on the safe speed limits corresponding to all risk calculation units. The preset compensation coefficient is obtained by taking the minimum value of the first ratio and 1; the first ratio is the ratio of the radius of curvature corresponding to the current risk calculation unit to the preset reference radius of curvature.

[0104] This invention can pre-calculate the corresponding safe speed limit for each risk calculation unit based on the road coupling risk index value. The safe speed limit can be calculated through the following steps:

[0105] in, This indicates the safe speed limit corresponding to the current risk calculation unit; This represents the road coupling risk index value corresponding to the current risk calculation unit; This indicates the preset base speed limit value; η represents the radius of curvature of the current risk calculation unit; η represents the preset risk index power, which is a non-linear index, usually taken as 1.5; This represents the reference radius of curvature, which is usually taken as 200 meters.

[0106] This invention can construct a safe speed limit map based on the safe speed limits corresponding to all risk calculation units. The final generated safe speed map contains safe speed limits across multiple grids. For a smooth transition, Gaussian filtering can be used to obtain a filtered safe speed map. The safe speed map may also include high-risk area markers. and suggested vehicle speed trajectory Where Vt is the suggested vehicle speed at time t:

[0107] in, For Gaussian weights; ∑ =1; This is a safe speed map; in practical applications, the safe speed limit corresponding to the target risk calculation unit can be determined from the pre-constructed safe speed limit map, so as to perform drive control on the vehicle based on the safe speed limit.

[0108] This invention constructs a safe speed map for each area within a drivable region. The system can complete risk assessment of complex scenarios within 0.15 seconds, with a warning lead time of 2-3 seconds. This is 5.2 times more effective than the traditional threshold method for early warning, fundamentally solving the technical bottleneck of "passive response" in speed control and improving the accuracy of risk terrain identification to 93.7%.

[0109] This invention can identify key safety risks such as the status of vehicles ahead, the depth of obstacles, and the degree of road slipperiness. It solves the key safety blind spot that traditional vehicle speed control systems cannot perceive micro-risks. When a sharp bend and a slippery road surface are detected ahead, the system can issue an early warning and automatically adjust the vehicle speed limit, reducing the risk of loss of control by 63% and effectively avoiding safety hazards caused by excessive speed.

[0110] This invention automatically adjusts the safe speed limit based on road risk level and vehicle status, eliminating the need for frequent manual adjustments by the driver and significantly reducing the operational difficulty and cognitive load of long-distance driving. Real-world test data shows that in complex road conditions, the system simplifies driving operations by 58% and increases user satisfaction to 91%, making it particularly suitable for highway and mountain road driving scenarios.

[0111] In one optional implementation, the vehicle control strategy further includes an air conditioning compressor power adjustment strategy, a torque distribution strategy, a braking coefficient preload strategy, and an energy recovery adjustment strategy. Besides limiting the safe speed, it can also determine the target load reduction level corresponding to the target risk calculation unit based on a preset mapping relationship between different road coupling risk index value ranges and different air conditioning power reduction levels. The air conditioning compressor power is adjusted based on the target load reduction level, and the target load reduction level is positively correlated with the road coupling risk index value range. A corresponding torque distribution strategy is determined based on the road feature information and inertial measurement data marked by the target risk calculation unit. The braking pressure loading level corresponding to the target risk calculation unit is determined based on a preset mapping relationship between different road coupling risk index value ranges and different braking pressure loading levels, and the braking pressure loading level is positively correlated with the road coupling risk index value range. Finally, the energy recovery intensity corresponding to the target risk calculation unit is determined based on a preset mapping relationship between different road coupling risk index value ranges and different energy recovery intensities, and the energy recovery intensity is negatively correlated with the road coupling risk index value range.

[0112] This invention takes into account that the road coupling risk index directly reflects the degree of danger of the current driving scenario of the vehicle. As the core electrical load of the vehicle, the power consumption of the air conditioner will occupy the vehicle's power resources. When the road risk is high, the vehicle needs to prioritize the protection of core safety functions such as power performance, braking performance and steering performance. Therefore, it is necessary to design a dynamic adjustment strategy for air conditioning power based on risk level to address the problem of the disconnect between the traditional air conditioning system and safety control. In high-risk scenarios (such as sharp bends, slippery roads, and tunnel entrances / exits), the system automatically detects the risk level. When the risk index exceeds the threshold of 0.6, it triggers the air conditioning load reduction mechanism: Level 1 load reduction (risk index 0.6-0.7): reduces the air conditioning compressor power by 10%, prioritizing the powertrain response while maintaining basic comfort; Level 2 load reduction (risk index 0.7-0.8): reduces the air conditioning compressor power by 15-20%, disables non-critical functions such as seat heating / ventilation, and significantly increases the available power of the powertrain; Level 3 load reduction (risk index > 0.8): reduces the air conditioning compressor power by 25-30%, retaining only basic ventilation functions to maximize the performance of the braking and steering systems. The system gradually restores the air conditioning power after the risk is eliminated through a thermal load prediction model, avoiding sudden temperature changes. In slippery road tests, this strategy improved the vehicle's power response speed by 28% and shortened the braking distance by 4.2 meters. In high-temperature environments, through coordinated optimization with the powertrain system, overall vehicle energy consumption is reduced by 12.3%, while maintaining the cabin temperature within a comfortable range. In one optional implementation, a baseline value for a risk index threshold is set. When the risk index threshold exceeds a first baseline value (0.6), the load reduction ratio can be adjusted accordingly. The load reduction ratio is calculated as (risk index - 0.6) × 50%. For example, when the risk index = 0.8, the load reduction is (0.8 - 0.6) × 50% = 10%. This is just an example. This invention uses the positive correlation mapping between the road coupling risk index and the air conditioning load reduction level to actively reduce the power of the air conditioning compressor in high-risk scenarios, allocating more power / electric resources to core safety systems such as braking, steering, and vehicle stability. This avoids the lag in the response of safety systems caused by the high power consumption of the air conditioning, further improving driving safety in high-risk scenarios. This echoes the core purpose of the road coupling risk assessment mentioned above: maintaining normal air conditioning power in low-risk scenarios to ensure driving comfort; prioritizing safety in high-risk scenarios and appropriately reducing the load. This avoids sacrificing all comfort for safety and also prevents neglecting safety for comfort, achieving a precise match between the two. This solves the shortcomings of traditional risk assessment, which only stays at the level of "index output" and cannot be implemented as an actual control strategy.

[0113] This invention addresses vehicle stability issues on curves and low-traction surfaces by designing an adaptive torque distribution strategy based on challenging terrain. This strategy comprehensively considers factors such as road curvature, lateral acceleration, road surface friction coefficient, and vehicle load, dynamically adjusting the torque distribution ratio between the front and rear axles and the left and right wheels: Curve scenario optimization: Upon detecting a curve, the system pre-adjusts the torque distribution based on the curve radius and vehicle speed. For left turns, it appropriately increases the torque on the right front wheel and decreases the torque on the left rear wheel, utilizing torque vectoring to assist steering and improve steering response precision. In continuous S-curve tests, this strategy reduced vehicle roll angle by 42% and improved passenger comfort by 28%. Low-traction road surface optimization: On wet, icy, or gravel surfaces, the system automatically reduces total drive torque by 15-20% and allocates torque towards wheels with better traction. For example, when the right side of the road is slippery, the system reduces right wheel torque and increases left wheel torque to prevent the vehicle from veering to the right. In icy and snowy road tests, this strategy reduced the risk of skidding by 58% and the number of times the vehicle lost control by 76%. Emergency obstacle avoidance optimization: When an emergency obstacle is detected ahead that needs to be avoided, the system instantly adjusts the torque distribution to maximize the use of tire adhesion. In an emergency obstacle avoidance test at a speed of 120 km / h, this strategy increased the obstacle avoidance success rate by 35% and reduced the risk of rollover by 62%. In an optional implementation, when the risk index threshold is greater than the second benchmark value (0.5), the reduction in rear axle torque can be calculated by torque offset = (risk index - 0.5) × 40%. For example, when the risk index = 0.75, the reduction in rear axle torque is (0.75 - 0.5) × 40% = 10%, which is only an example.

[0114] To shorten braking response time, this invention employs a risk-prediction-based braking preloading strategy. This strategy analyzes the risk level of the road ahead and the distance to obstacles to pre-establish braking system pressure: Level 1 preloading (risk index 0.4-0.5): When a medium-risk area (such as a gentle curve or slightly slippery road) is detected ahead, a base braking pressure of 0.5-1.0 MPa is established, shortening the response time by 15%; Level 2 preloading (risk index 0.5-0.7): When a high-risk area (such as a sharp curve or a flooded road) is detected ahead, a medium braking pressure of 1.0-1.5 MPa is established, shortening the response time by 25%; Level 3 preloading (risk index > 0.7): When an extremely high-risk area (such as a sharp curve + slippery road or sudden braking ahead) is detected, a high braking pressure of 1.5-2.0 MPa is established, shortening the response time by 30-35%. The system uses a progressive preloading mechanism to avoid sudden braking sensations. At a vehicle speed of 100km / h, when a sharp curve is detected 150 meters ahead, the system begins preloading 3 seconds in advance, reducing the braking response time from the traditional 450ms to 300ms and the braking distance by 5-8 meters, significantly improving driving safety. In one optional implementation, when the risk index threshold is greater than the third benchmark value (0.4), the preload pressure can be calculated as preload pressure = risk index × 2.0MPa. For example, when the risk index = 0.85, the preload pressure is 0.85 × 2.0 = 1.7MPa, which is only an example.

[0115] This invention addresses the conflict between energy recovery and vehicle stability in electric vehicles by designing a dynamic adjustment strategy for energy recovery intensity based on risk levels: Low-risk scenarios (risk index < 0.3): High-intensity energy recovery (recovery intensity 0.25-0.3g) is used in low-risk scenarios such as straight roads and dry surfaces to maximize energy recovery efficiency and improve driving range; Medium-risk scenarios (risk index 0.3-0.5): Medium-intensity energy recovery (recovery intensity 0.15-0.2g) is used in medium-risk scenarios such as gentle curves and slight slopes to balance safety and energy efficiency; High-risk scenarios (risk index > 0.5): Low-intensity or zero-intensity energy recovery (recovery intensity < 0.1g) is used in high-risk scenarios such as sharp curves and slippery surfaces to prioritize vehicle stability and handling; This strategy monitors vehicle lateral acceleration and tire slip ratio in real time, and instantly reduces the energy recovery intensity when a decrease in vehicle stability is detected. In mountain road tests, this strategy improved energy recovery efficiency by 18% while maintaining vehicle stability within a safe range. In one optional implementation, when the risk index threshold is greater than the fourth baseline value (0.3), the recovery intensity can be calculated as recovery intensity = max(0, 0.3 - risk index) × 0.3g. For example, when the risk index = 0.6, the recovery intensity = max(0, 0.3 - 0.6) × 0.3 = 0. This is just an example. Based on the above progressive design, the shortcomings of discrete classification can be solved, that is, the 5-level classification cannot distinguish the subtle differences in "high risk" (such as a sharp bend 80 meters ahead vs. (120-meter sharp bend) The necessity of continuous quantification: The power system requires precise control parameters. For example, a risk index of 0.72 requires a 12% reduction in air conditioning load, while 0.85 requires a 20% reduction; a risk index of 0.65 requires a braking preload of 1.3 MPa, while 0.9 requires 1.8 MPa; Spatiotemporal continuity assurance: Spatial continuity: The rate of change of risk index between adjacent grids is limited to ±0.1 / 5 meters to avoid abrupt changes in control parameters; Temporal continuity: The rate of change of risk index over time is limited to ±0.05 / 100 ms to ensure smooth control; Example: When a vehicle enters a curve from a straight road, the risk index smoothly transitions from 0.2 to 0.85, and the air conditioning power linearly decreases from 100% to 80%, which is only an example.

[0116] When high-risk scenarios are detected (such as continuous curves and heavy vehicle loads), this invention not only provides visual warnings, but also automatically controls the air conditioning power, torque distribution and active suspension parameters to provide all-round driving safety. Under extreme test conditions, the system successfully prevented 85% of potential loss of control accidents, shortened the braking distance by an average of 18%, and significantly improved driving safety.

[0117] To enhance the personalization and adaptability of control strategies, this invention introduces an adaptive learning mechanism: driver habit learning. By analyzing historical driver data (such as cornering speed, braking timing, and acceleration habits), a personalized driving model is established. While ensuring safety, the control strategy is adjusted according to driver preferences. For example, for aggressive drivers, comfort constraints are appropriately relaxed to improve response speed; for conservative drivers, safety constraints are strengthened to provide a smoother driving experience. Scene adaptive optimization: The system continuously learns the control effects under different scenarios and automatically optimizes strategy parameters. On continuous curved road sections, it can learn the optimal torque distribution ratio; on slippery roads, it can learn the most effective air conditioning load reduction. Through online learning, the accuracy of the control strategy improves by 2-3% per month. Multi-objective weight adaptation: Based on driving mode (economy mode, comfort mode, sport mode) and environmental conditions (temperature, road conditions, traffic density), the weights of safety, comfort, and energy efficiency are dynamically adjusted. In high-temperature environments, the energy efficiency weight is appropriately increased; in complex road conditions, the safety weight is significantly increased.

[0118] This invention provides personalized safe speed recommendations based on driver habits, vehicle load, and road conditions, including recommended speeds for curves, gradient adjustment prompts, and comfort optimization suggestions, optimizing the driving experience while ensuring safety. When a tunnel or bridge is detected ahead, the system adjusts the vehicle speed and suspension parameters in advance to reduce bumps and improve passenger comfort by 31%.

[0119] Based on comprehensive evaluation results, this invention implements multi-objective collaborative optimization of driving safety, comfort, and energy efficiency: Personalized speed recommendations: Based on driver habits, vehicle load, and road conditions, the system provides suggested speeds for curves, gradient adjustment prompts, and comfort optimization suggestions through a human-machine interface (HMI), improving average driving efficiency by 8.2% while ensuring safety; Active suspension pre-adjustment: On continuous curved road sections, the system pre-adjusts suspension damping parameters based on a risk terrain model, reducing roll angle by 42% and improving passenger comfort by 28%; Adaptive driving style: The system learns driver operating habits and provides personalized acceleration / braking curve suggestions while maintaining a safety margin, reducing unnecessary rapid acceleration and braking; Energy consumption optimization management: In high-temperature environments, intelligent thermal... The management strategy coordinates the load of the air conditioning and powertrain systems, reducing overall vehicle energy consumption by 12.3%. On long downhill sections, the energy recovery strategy is optimized, improving energy recovery efficiency by 18% while ensuring braking safety. Through the above coordinated control, the system reduced the risk of speed-related accidents by 58%, improved driving comfort score by 31%, and reduced overall vehicle energy consumption by an average of 9.6% in real-world vehicle tests. The overall system response time is controlled within 20ms, achieving a complete closed loop from "environmental perception - risk modeling - vehicle speed decision - power coordination - effect feedback," providing comprehensive protection for the safe driving of intelligent connected vehicles.

[0120] To ensure system reliability, this invention also employs multiple safety assurance mechanisms: Functional safety level: Designed according to ISO 26262 (Road vehicles - Functional safety) and ASIL-D (Automotive Safety Integrity Level D), all critical control commands undergo double verification to ensure command correctness. The control algorithm includes an anomaly detection module capable of identifying abnormal situations such as sensor malfunctions and communication interruptions. Redundant control channels are provided: a primary control channel and a backup safety channel are configured. The primary channel handles refined collaborative control, while the backup channel provides basic safety control functions (such as emergency braking and basic torque distribution) when the primary channel fails. The two channels monitor each other's status in real time via heartbeat signals. A gradual degradation strategy is employed to mitigate partial system failures, rather than abrupt failures. For example, when a vision sensor fails, the system degrades to an inertial measurement unit-only mode, maintaining basic risk assessment capabilities. When the communication bus fails, each subsystem enters a safety mode, maintaining minimal collaborative control. Real-world testing validated the module, which reduced the risk of speed-related accidents by 58% in a 150,000 km comprehensive test. The system achieved a 31% improvement in driving comfort and a 9.6% reduction in overall vehicle energy consumption. Under extreme testing conditions (such as heavy rain, sharp curves, and sudden braking), it successfully prevented 85% of potential loss-of-control accidents, demonstrating its superior safety performance and reliability. It not only solves the problem of independent control in traditional powertrain systems but also achieves a comprehensive value of "safety-comfort-energy efficiency" through a risk-driven collaborative optimization mechanism, representing the development direction of intelligent connected vehicle powertrain control technology. Technically, it achieves a comprehensive improvement in perception accuracy, response speed, and system robustness. In terms of user experience, it significantly reduces the driver's operational burden and safety anxiety. In terms of commercial value, it provides an innovative solution for differentiated competition in intelligent connected vehicles and is expected to become a standard feature in mid-to-high-end intelligent vehicles.

[0121] This embodiment also provides a vehicle, as shown in FIG5. The vehicle includes a controller 501, which includes a memory and a processor. The memory and the processor are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to perform the vehicle control method, which will not be described in detail here.

[0122] This embodiment also provides a vehicle control device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0123] This embodiment provides a vehicle control device, as shown in Figure 6, including: a data acquisition module 601, used to acquire inertial measurement data and forward vision data of the vehicle; a data fusion module 602, used to fuse the inertial measurement data and forward vision data to obtain fused perception data; an environment perception module 603, used to input the fused perception data into a multimodal neural network model to obtain the environment perception result output by the model, the environment perception result including at least a drivable area semantic map; a risk index assessment module 604, used to divide the current drivable area in the drivable area semantic map into multiple risk calculation units, and based on the environment perception result and inertial measurement data, assess the road coupling risk index value of each risk calculation unit, wherein the road coupling risk index value is used to quantify the driving risk level of the road under the coupling effect of multiple factors; and a vehicle control module 605, used to generate a corresponding vehicle control strategy based on the road coupling risk index value corresponding to the target risk calculation unit into which the vehicle is about to enter, and control the vehicle driving based on the vehicle control strategy.

[0124] In some optional implementations, the risk index assessment module 604 includes: a feature extraction unit, used to extract features from inertial measurement data and environmental perception results of each risk calculation unit to obtain a risk assessment feature set; a weight adjustment unit, used to adjust the feature weights corresponding to each feature vector in the risk assessment feature set based on inertial measurement data and road feature information marked for the current risk calculation unit in the semantic map of the drivable area; and an index value calculation unit, used to perform weighted fusion of each feature vector based on the feature weights corresponding to each feature vector to obtain the road coupling risk index value corresponding to the current risk calculation unit.

[0125] In some optional implementations, the risk assessment feature set includes at least one of a traffic state feature vector set, a road feature vector set, and a vehicle feature vector set. The index value calculation unit includes: an index value calculation subunit, used to calculate the road risk index value, vehicle risk index value, and / or traffic risk index value respectively for the traffic state feature vector set, road feature vector set, and / or vehicle feature vector set corresponding to the current risk calculation unit, based on the feature weights corresponding to the feature vectors in each feature vector set; and a weighted fusion subunit, used to perform weighted fusion of the road risk index value, vehicle risk index value, and / or traffic risk index value to obtain the road coupling risk index value corresponding to the current risk calculation unit.

[0126] In some optional implementations, the environmental perception results also include obstacle depth estimation results and vehicle status information ahead. The environmental perception module 803 includes: a feature extraction unit, used to input the fused perception data into a multimodal neural network model, so that the model performs multi-level feature extraction and fusion on the spatiotemporally aligned forward visual data to obtain a multi-scale visual feature map; a feature operation unit, used to perform upsampling and stitching operations on the multi-scale visual feature map to obtain a fused visual feature map; and a result output unit, used to perform obstacle depth estimation, drivable area semantic segmentation, and vehicle status detection ahead based on the fused visual feature map to obtain obstacle depth estimation results, drivable area semantic map, and vehicle status information ahead.

[0127] In some optional implementations, the vehicle control device includes: a temporal feature extraction module for extracting temporal features from inertial measurement data to obtain temporal dynamic features; an information fusion module for fusing a drivable area semantic map, obstacle depth estimation results, and vehicle state information ahead to obtain fused visual features; a spatial dependency modeling module for performing spatial dependency modeling on the fused visual features to obtain enhanced visual features; a temporal dynamic modeling module for modeling vehicle dynamic evolution on the temporal dynamic features to generate temporal enhanced features; a feature transformation module for performing multilayer perceptron transformation on the temporal enhanced features to obtain inertial measurement risk features; a cross-modal fusion module for performing cross-modal fusion of the fused visual features and inertial measurement risk features through a nonlinear attention mechanism to obtain cross-modal features; a risk level determination module for determining the risk level of each drivable area in the drivable area semantic map based on the cross-modal features; and a graded warning module for providing graded warnings based on the risk level of each drivable area the vehicle is about to enter.

[0128] In some optional implementations, the vehicle control device further includes: an information acquisition module for acquiring environmental data and vehicle status information of the current driving environment; an environmental correction coefficient determination module for determining the corresponding environmental correction coefficient based on the current driving environmental data; a vehicle status correction coefficient determination module for determining the corresponding vehicle status correction coefficient based on the vehicle status information; a final risk index calculation module for calculating the final risk index value based on the road coupling risk index value, environmental correction coefficient, and vehicle status correction coefficient corresponding to the current risk calculation unit; and a value update module for updating the road coupling risk index value corresponding to the current risk calculation unit to the final risk index value.

[0129] In some optional implementations, the vehicle control module 605 includes: a speed limit map construction unit, used to determine the safe speed limit corresponding to the target risk calculation unit from a pre-constructed safe speed limit map, so as to perform drive control on the vehicle based on the safe speed limit. The safe speed limit map is constructed by the following steps: calculating the preset risk index power of the road coupling risk index value corresponding to each risk calculation unit to obtain a first exponential product, and taking the complement value of the first exponential product; multiplying the preset basic speed limit value, the complement value and the preset compensation coefficient to obtain the safe speed limit corresponding to each risk calculation unit; and constructing a safe speed limit map based on the safe speed limits corresponding to all risk calculation units. The preset compensation coefficient is obtained by taking the minimum value of a first ratio and 1. The first ratio is the ratio of the radius of curvature corresponding to the current risk calculation unit to the preset reference radius of curvature.

[0130] In some optional implementations, the vehicle control module 605 includes: determining a target load reduction level corresponding to the target risk calculation unit based on a preset mapping relationship between different road coupling risk index value ranges and different air conditioning power load reduction levels; adjusting the air conditioning compressor power based on the target load reduction level, wherein the target load reduction level is positively correlated with the road coupling risk index value range; determining a corresponding torque distribution strategy based on the road feature information and inertial measurement data marked by the target risk calculation unit; determining a braking pressure loading level corresponding to the target risk calculation unit based on a preset mapping relationship between different road coupling risk index value ranges and different braking pressure loading levels, wherein the braking pressure loading level is positively correlated with the road coupling risk index value range; and determining an energy recovery intensity corresponding to the target risk calculation unit based on a preset mapping relationship between different road coupling risk index value ranges and different energy recovery intensities, wherein the energy recovery intensity is negatively correlated with the road coupling risk index value range.

[0131] In some optional implementations, the risk calculation unit is a risk calculation grid, which divides the current drivable area in the drivable area semantic map into multiple risk calculation units, including: determining the region of interest of the cone ahead based on the vehicle's driving direction, and taking the region of interest as the current drivable area; adjusting the size of the risk calculation grid according to the current vehicle speed, so as to divide the current drivable area in the drivable area semantic map into multiple risk calculation grids based on the size, and the vehicle speed is positively correlated with the size of the grid.

[0132] The vehicle control device provided in this embodiment of the invention can execute the vehicle control method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as in the corresponding embodiments described above, and will not be repeated here.

[0133] Figure 7 is a schematic diagram of the structure of a controller provided in an embodiment of the present invention.

[0134] Referring specifically to Figure 7, a schematic diagram of a structure suitable for implementing the controller in an embodiment of the present invention is shown below. The controller may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 702 or a program loaded from memory 708 into random access memory (RAM) 703. The RAM 703 also stores various programs and data required for controller operation. The processor 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0135] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 707 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; memory 708 including, for example, magnetic tape, hard disk, etc.; and communication devices 709. Communication device 709 allows the controller to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 shows a controller with various devices, it should be understood that it is not required to implement or have all the devices shown, and more or fewer devices may be implemented alternatively.

[0136] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 709, or installed from a memory 708, or installed from a ROM 702. When the computer program is executed by the processor 701, it performs the functions defined in the vehicle control method of the embodiments of the present invention.

[0137] The controller shown in Figure 7 is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0138] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the vehicle control method shown in the above embodiments is implemented.

[0139] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0140] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A vehicle control method, characterized in that, The method includes: acquiring inertial measurement data and forward-looking visual data of the vehicle; fusing the inertial measurement data and forward-looking visual data to obtain fused perception data; inputting the fused perception data into a multimodal neural network model to obtain the environmental perception result output by the model, wherein the environmental perception result includes at least a drivable area semantic map; dividing the currently drivable area in the drivable area semantic map into multiple risk calculation units, and evaluating the road coupling risk index value for each risk calculation unit based on the environmental perception result and inertial measurement data, wherein the road coupling risk index value is used to quantify the driving risk level of the road under the coupling effect of multiple factors; generating a corresponding vehicle control strategy based on the road coupling risk index value corresponding to the target risk calculation unit into which the vehicle is about to enter, and controlling the vehicle's driving based on the vehicle control strategy.

2. The method according to claim 1, characterized in that, The step of evaluating the road coupling risk index value for each risk calculation unit based on the environmental perception results and inertial measurement data includes: extracting features from the inertial measurement data and the environmental perception results of each risk calculation unit to obtain a risk assessment feature set; adjusting the feature weights corresponding to each feature vector in the risk assessment feature set based on the inertial measurement data and the road feature information marked for the current risk calculation unit in the semantic map of the drivable area; and performing weighted fusion of the feature vectors based on the feature weights corresponding to each feature vector to obtain the road coupling risk index value corresponding to the current risk calculation unit.

3. The method according to claim 2, characterized in that, The risk assessment feature set includes at least one of a traffic state feature vector set, a road feature vector set, and a vehicle feature vector set. The step of weighted fusion of the feature vectors based on their corresponding feature weights to obtain the road coupling risk index value corresponding to the current risk calculation unit includes: calculating the road risk index value, vehicle risk index value, and / or traffic risk index value for the traffic state feature vector set, road feature vector set, and / or vehicle feature vector set corresponding to the current risk calculation unit, according to the feature weights corresponding to the feature vectors in each feature vector set; and weighted fusion of the road risk index value, the vehicle risk index value, and / or the traffic risk index value to obtain the road coupling risk index value corresponding to the current risk calculation unit.

4. The method according to claim 1, characterized in that, The environmental perception results also include obstacle depth estimation results and vehicle status information ahead. The step of inputting the fused perception data into a multimodal neural network model to obtain the model's output environmental perception results includes: inputting the fused perception data into the multimodal neural network model so that the model performs multi-level feature extraction and fusion on the spatiotemporally aligned forward-looking visual data to obtain a multi-scale visual feature map; performing upsampling and stitching operations on the multi-scale visual feature map to obtain a fused visual feature map; and based on the fused visual feature map, performing obstacle depth estimation, drivable area semantic segmentation, and vehicle status detection ahead to obtain obstacle depth estimation results, a drivable area semantic map, and vehicle status information ahead.

5. The method according to claim 4, characterized in that, The method further includes: extracting temporal features from the inertial measurement data to obtain temporal dynamic features; fusing the drivable area semantic map, obstacle depth estimation results, and vehicle state information ahead to obtain fused visual features; performing spatial dependency modeling on the fused visual features to obtain enhanced visual features; performing vehicle dynamic evolution modeling on the temporal dynamic features to generate temporal enhanced features; performing multilayer perceptron transformation on the temporal enhanced features to obtain inertial measurement risk features; performing cross-modal fusion on the fused visual features and inertial measurement risk features through a nonlinear attention mechanism to obtain cross-modal features; determining the risk level of each drivable area in the drivable area semantic map based on the cross-modal features; and providing graded warnings based on the risk level of each drivable area the vehicle is about to enter.

6. The method according to claim 2 or 3, characterized in that, The method further includes: acquiring environmental data and vehicle status information of the current driving of the vehicle; determining the corresponding environmental correction coefficient based on the current driving environmental data; determining the corresponding vehicle status correction coefficient based on the vehicle status information; calculating the final risk index value based on the road coupling risk index value, environmental correction coefficient, and vehicle status correction coefficient corresponding to the current risk calculation unit; and updating the road coupling risk index value corresponding to the current risk calculation unit to the final risk index value.

7. The method according to claim 1, characterized in that, The vehicle control strategy includes at least a safe speed limit. The generation of a corresponding vehicle control strategy based on the road coupling risk index value corresponding to the target risk calculation unit into which the vehicle is about to enter includes: determining the safe speed limit corresponding to the target risk calculation unit from a pre-constructed safe speed limit map, and performing drive control on the vehicle based on the safe speed limit. The safe speed limit map is constructed through the following steps: calculating a preset risk index power of the road coupling risk index value corresponding to each risk calculation unit to obtain a first exponential product, and taking the complement value of the first exponential product; multiplying a preset base speed limit value, the complement value, and a preset compensation coefficient to obtain the safe speed limit corresponding to each risk calculation unit; and constructing a safe speed limit map based on the safe speed limits corresponding to all risk calculation units. The preset compensation coefficient is obtained by taking the minimum value of a first ratio and 1; the first ratio is the ratio of the radius of curvature corresponding to the current risk calculation unit to a preset reference radius of curvature.

8. The method according to claim 1 or 7, characterized in that, The vehicle control strategy also includes an air conditioning compressor power adjustment strategy, a torque distribution strategy, a braking coefficient preloading strategy, and an energy recovery adjustment strategy. The generation of the corresponding vehicle control strategy based on the road coupling risk index value corresponding to the target risk calculation unit the vehicle is about to enter includes: determining the target load reduction level corresponding to the target risk calculation unit based on a preset mapping relationship between different road coupling risk index value ranges and different air conditioning power reduction levels; adjusting the air conditioning compressor power based on the target load reduction level, where the target load reduction level is positively correlated with the road coupling risk index value range; determining the corresponding torque distribution strategy based on the road feature information and inertial measurement data marked by the target risk calculation unit; determining the braking pressure loading level corresponding to the target risk calculation unit based on a preset mapping relationship between different road coupling risk index value ranges and different braking pressure loading levels, where the braking pressure loading level is positively correlated with the road coupling risk index value range; and determining the energy recovery intensity corresponding to the target risk calculation unit based on a preset mapping relationship between different road coupling risk index value ranges and different energy recovery intensities, where the energy recovery intensity is negatively correlated with the road coupling risk index value range.

9. The method according to claim 1, characterized in that, The risk calculation unit is a risk calculation grid, which divides the current drivable area in the drivable area semantic map into multiple risk calculation units. This includes: determining the region of interest (ROI) of the cone shape ahead based on the vehicle's driving direction, and using the ROI as the current drivable area; adjusting the size of the risk calculation grid based on the current vehicle speed, so as to divide the current drivable area in the drivable area semantic map into multiple risk calculation grids based on the size, wherein the vehicle speed is positively correlated with the size of the grid.

10. A vehicle control device, characterized in that, The device includes: a data acquisition module for acquiring inertial measurement data and forward-looking visual data of the vehicle; a data fusion module for fusing the inertial measurement data and forward-looking visual data to obtain fused perception data; an environment perception module for inputting the fused perception data into a multimodal neural network model to obtain the environment perception result output by the model, wherein the environment perception result includes at least a drivable area semantic map; a risk index assessment module for dividing the current drivable area in the drivable area semantic map into multiple risk calculation units, and assessing the road coupling risk index value of each risk calculation unit based on the environment perception result and inertial measurement data, wherein the road coupling risk index value is used to quantify the driving risk level of the road under the coupling effect of multiple factors; and a vehicle control module for generating a corresponding vehicle control strategy based on the road coupling risk index value corresponding to the target risk calculation unit into which the vehicle is about to enter, and controlling the vehicle's driving based on the vehicle control strategy.

11. A vehicle, characterized in that, The vehicle includes a controller, which includes a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the vehicle control method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the vehicle control method according to any one of claims 1 to 9.

Images