Vehicle control decision-making method and device based on perceptual fusion technology, and electronic equipment

Abstract

The invention relates to the technical field of intelligent driving, and provides a vehicle control decision-making method and device based on the perceptual fusion technology and electronic device.The vehicle control decision-making method based on the perceptual fusion technology comprises the steps that first positioning information of a vehicle is obtained through data fusion of at least two complementary sensors, the complementary sensor comprises an inertial navigation system, a visual odometer and a laser radar odometer or odometer; correcting the first positioning information through a roadside assisted positioning technology to obtain second positioning information; state information of a target vehicle is acquired through a cellular Internet of Vehicles communication technology; fusing the environment sensing data collected by the vehicle-mounted sensor and the roadside equipment of the vehicle and the state information of the target vehicle, and constructing a global environment model; and making a control decision based on the second positioning information and the global environment model to realize safe vehicle distance keeping and collision avoidance. The method improves the perception, decision and control capabilities of the vehicle.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent driving, and particularly to a vehicle control decision-making method, device and electronic device based on perception fusion technology. Background Art

[0002] With the rapid development of autonomous driving technology, intelligent driving systems are increasingly widely used in ground transportation. Intelligent driving vehicles integrate multiple sensors (such as lidar, cameras, millimeter-wave radars, etc.) and advanced algorithms to achieve perception, decision-making and control of the surrounding environment. The progress of these technologies enables vehicles to drive safely and efficiently in complex traffic environments.

[0003] In existing intelligent driving technologies, vehicles usually rely on GPS and other satellite positioning systems to obtain position information. However, in certain specific scenarios, such as the underground mine scenarios of coal mines and tunnels, the availability of satellite signals is limited, resulting in the inability of vehicles to accurately locate. In addition, the pure inertial navigation system (INS) has a large cumulative error during long-term operation, and visual odometry and lidar SLAM are severely affected by light and dust, and lack roadside cooperation, making it difficult to achieve high-precision global environment modeling and real-time control decision-making. The direct consequence of this defect is that the vehicle's perception system cannot effectively identify obstacles and other traffic participants in the surrounding environment, thus increasing the risk of accidents. Summary of the Invention

[0004] The present invention provides a vehicle control decision-making method, device and electronic device based on perception fusion technology, which are used to solve the defect that the perception and decision-making capabilities of vehicles decline in an environment without satellite positioning signals in the prior art, and reduce the risk of accidents.

[0005] The present invention provides a vehicle control decision-making method based on perception fusion technology, including: obtaining first positioning information of the vehicle through data fusion of at least two complementary sensors, where the complementary sensors include: an inertial navigation system, visual odometry, lidar odometry or odometer; correcting the first positioning information through roadside assisted positioning technology to obtain second positioning information; obtaining the state information of the target vehicle through cellular vehicle-to-everything communication technology; fusing the environmental perception data collected by the vehicle-mounted sensors and roadside devices of the vehicle and the state information of the target vehicle to construct a global environment model; making a control decision based on the second positioning information and the global environment model to achieve safe distance keeping and collision avoidance.

[0006] According to the vehicle control decision-making method based on perception fusion technology provided by the present invention, the roadside auxiliary positioning technology includes at least one of the following: ultra-wideband positioning, geomagnetic matching positioning, and radio signal fingerprint positioning; the vehicle-mounted sensors include cameras, lidar, and millimeter-wave radars, and the roadside devices include cameras and lidar.

[0007] According to the vehicle control decision-making method based on perception fusion technology provided by the present invention, obtaining the first positioning information of the vehicle through data fusion of at least two complementary sensors includes: fusing the data of at least two complementary sensors through the Kalman filter or extended Kalman filter algorithm to eliminate noise and improve positioning accuracy.

[0008] According to the vehicle control decision-making method based on perception fusion technology provided by the present invention, fusing the environmental perception data collected by the vehicle-mounted sensors and roadside devices of the vehicle and the state information of the target vehicle includes: using a target tracking algorithm to match the target vehicle perceived by the vehicle with the state information of the target vehicle, and realizing data fusion through the Kalman filter or particle filter algorithm.

[0009] According to the vehicle control decision-making method based on perception fusion technology provided by the present invention, the global environmental model is a set of dynamic target states , where p i is the three-dimensional position coordinate of the i-th target, v i is the velocity vector of the i-th target, a i is the acceleration vector of the i-th target, N is the total number of targets perceived by the vehicle, and the target state is updated through a spatio-temporal synchronization algorithm.

[0010] According to the vehicle control decision-making method based on perception fusion technology provided by the present invention, making a control decision based on the second positioning information and the global environmental model includes:

[0011] Evaluating the collision risk based on the following formula:

[0012] ;

[0013] ;

[0014] where t collision represents the predicted time to collision, d stop represents the braking distance of the vehicle, d SV-TV represents the relative distance between the vehicle and the target vehicle, v TV represents the speed of the target vehicle, t react represents the driver or system reaction time, v SV represents the speed of the vehicle, a max represents the maximum acceleration of the vehicle;

[0015] Emergency braking is triggered when the following formula is satisfied:

[0016] ;

[0017] ;

[0018] where t thresh represents a pre-set collision time threshold, with a value range of 1 - 3 seconds, dynamically adjusted according to the scenario.

[0019] According to the vehicle control decision-making method based on perception fusion technology provided by the present invention, the first positioning information of the vehicle is obtained through data fusion of at least two complementary sensors, including: adopting a multi-sensor fusion positioning algorithm based on federated learning, where the at least two sensors independently train local positioning models, and aggregate global model parameters through a federated learning framework. The aggregation weight of the federated learning is determined according to the following formula:

[0020] ;

[0021] where n represents the number of sensors, W k represents the local positioning model parameters of the k-th sensor, and α k represents the weight ratio of the k-th sensor in the aggregation of the federated learning model after normalization processing. The calculation formula is as follows:

[0022] ;

[0023] where C i is the confidence score of the i-th sensor, and C k is the confidence score of the k-th sensor, calculated based on historical errors and current environmental parameters. The calculation formula is:

[0024] ;

[0025] where α and β are adjustment coefficients, dynamically adjusted according to the real-time scenario, and satisfy that the sum of α and β is 1, and σ k 2 is the historical positioning error variance of the k-th sensor, calculated by sliding window statistics, and E k is the environmental parameter score. The calculation formula is:

[0026] ;

[0027] where D k represents the dust concentration of the current scenario, L k is the light intensity, is the water mist density, and γ 1, γ 2 , γ 3 is a preset environmental weight coefficient and satisfies γ 1 + γ 2 + γ 3 = 1.

[0028] According to the vehicle control decision-making method based on the perception fusion technology provided by the present invention, the method further includes: simulating sensor noise in an extreme environment through a generative adversarial network, training a robust fusion model, and real-time detecting abnormal patterns in the input data. If adversarial interference is detected, a redundant sensor data replacement mechanism is activated.

[0029] The present invention also provides a vehicle control decision-making device based on the perception fusion technology, which is characterized by including: a first acquisition module configured to obtain the first positioning information of the vehicle itself through data fusion of at least two complementary sensors, and the complementary sensors include: an inertial navigation system, a visual odometer, a lidar odometer or an odometer; a calibration module configured to calibrate the first positioning information through a roadside assisted positioning technology to obtain a second positioning information; a second acquisition module configured to obtain the status information of a target vehicle through a cellular vehicle-to-everything communication technology; a fusion module configured to fuse the environmental perception data collected by in-vehicle sensors and roadside devices of the vehicle itself and the status information of the target vehicle to construct a global environmental model; and a control module configured to make a control decision based on the second positioning information and the global environmental model to achieve safe distance keeping and collision avoidance.

[0030] The vehicle control decision-making method, device and electronic device based on the perception fusion technology provided by the present invention propose a new intelligent driving solution, which can fuse the data of the vehicle's own sensors with the data obtained by other positioning and perception technologies in an environment without satellite positioning signals, and improve the vehicle's perception, decision-making and control capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0032] Figure 1 is a flowchart of the vehicle control decision-making method based on the perception fusion technology provided by the present invention.

[0033] Figure 2 is a schematic diagram of the process of fusing the self-positioning technology and the environmental perception information provided by the present invention.

[0034] Figure 3 It is a schematic structural diagram of the vehicle control decision-making method based on the perception fusion technology provided by the present invention.

[0035] Figure 4 It is a schematic structural diagram of the electronic device provided by the present invention. Detailed implementation manners

[0036] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0037] Unless otherwise defined, the technical terms or scientific terms used in the present invention shall have the ordinary meanings understood by those of ordinary skill in the art to which the present invention pertains. The "first", "second" and similar terms used in the present invention do not denote any order, quantity or importance, but are only used to distinguish different components. Similarly, the terms such as "a", "an" or "the" do not denote a quantity limitation, but mean that there is at least one. The terms such as "comprising" or "including" mean that the elements or objects appearing before this term cover the elements or objects listed after this term and their equivalents, without excluding other elements or objects. The terms such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0038] The terms related to the present invention are briefly explained below.

[0039] Inertial Navigation System (INS): Measures the acceleration and angular velocity of a vehicle using accelerometers and gyroscopes, and calculates the position and attitude of the vehicle through integral operations.

[0040] Visual Odometry: Estimates the motion trajectory of a vehicle by analyzing a sequence of images captured by a camera.

[0041] LiDAR Odometry: Estimates the motion trajectory of a vehicle by analyzing the point cloud data scanned by a LiDAR.

[0042] Ultra-Wideband (UWB) positioning: Achieves high-precision positioning based on the measurement of the Time of Flight (TOF) or Angle of Arrival (AOA) of UWB signals.

[0043] Geomagnetic matching positioning: The geomagnetic field intensity is measured by a magnetometer on the vehicle and matched with a pre-constructed geomagnetic map to determine the vehicle's position.

[0044] Radio signal fingerprint positioning: The radio signal strengths of Wi-Fi, Bluetooth, etc. around are collected and matched with a pre-constructed signal fingerprint map to determine the vehicle's position.

[0045] Odometer: The driving distance of the vehicle is estimated by measuring the number of rotations of the wheels.

[0046] The present invention can be applied to enclosed traffic scenarios without satellite positioning signals (Global Navigation Satellite System shielding rate ≥ 99%), typically including:

[0047] Underground mine roadways: horizontal / inclined / spiral roadways, with a curvature radius ≥ 15 m and a dynamic range of dust concentration of 50 - 1000 μg / m³

[0048] Ultra-long tunnel networks: with a length ≥ 3 km, an illuminance ≤ 20 lux, and periodic ventilation disturbance airflows (wind speed 2 - 5 m / s)

[0049] Military / civilian bunkers: multi-layer reinforced concrete structures, with a metal reflection surface ratio > 40% and significant electromagnetic shielding effects

[0050] Scene common constraints:

[0051] Absolute positioning failure: Satellite signals such as GPS / Beidou are completely unavailable;

[0052] Relative positioning interference: Multipath effects cause UWB / TDoA ranging errors > 30 cm;

[0053] Perception-communication coupling: The combined effect of dust / water mist causing LiDAR attenuation and wireless signal scattering.

[0054] The following combines Figures 1-4 to describe the vehicle control decision-making method, device, and electronic device of the present invention based on perception fusion technology.

[0055] Figure 1 is a schematic flow diagram of the vehicle control decision-making method based on perception fusion technology provided by the present invention. As Figure 1 shown, the method includes the following:

[0056] Step 101: Obtain the first positioning information of the vehicle through data fusion of at least two complementary sensors.

[0057] In this embodiment, the sources of sensor data may include at least two of the following: inertial navigation system, visual odometer, lidar odometer, or odometer. By using technologies such as inertial navigation system (INS), visual inertial odometer (VIO), and ground feature matching, it can be ensured that the vehicle can still achieve accurate positioning in the absence of satellite signals. By fusing various sensor data such as its own inertial navigation, visual odometer, and lidar odometer, and using algorithms such as Kalman Filter and Extended Kalman Filter (EKF) for data fusion, the positioning accuracy and robustness can be improved. Among them, Kalman Filter can fuse multi-sensor data, eliminate noise and errors, and improve positioning accuracy, and Extended Kalman Filter (EKF) can process non-linear sensor data to further improve positioning robustness.

[0058] Step 102: Correct the first positioning information through roadside assisted positioning technology to obtain the second positioning information.

[0059] In this embodiment, roadside units (RSUs) can deploy positioning beacons such as UWB, Bluetooth, Wi-Fi, and geomagnetic matching. By receiving these beacon signals, the vehicle can perform assisted positioning using methods such as triangulation or fingerprint matching to further improve positioning accuracy.

[0060] Step 103: Obtain the status information of the target vehicle through cellular vehicle-to-everything communication technology.

[0061] In this embodiment, the status information of the target vehicle may include the position information of the target vehicle and motion states (such as speed, acceleration, braking signal), etc., which can predict the future behavior of the TV. Although vehicle-to-vehicle (V2V) communication technology can make up for the deficiencies of single-vehicle perception to a certain extent, in the scenario without satellite positioning signals, the stability and reliability of V2V communication also face challenges, so it cannot rely solely on V2V communication technology. In addition, the available frequency bands can be monitored in real time through cognitive radio technology, and high signal-to-noise ratio frequency bands can be preferentially allocated for vehicle-to-vehicle communication. Adaptive modulation and coding (AMC) is used to dynamically adjust transmission parameters to ensure that the communication delay ≤ 10 ms and the packet loss rate ≤ 0.1%. When the cellular network signal is weak, it can also automatically switch to infrared or ultrasonic communication and expand the coverage range through multi-hop relay technology.

[0062] Step 104: Fuse the environmental perception data collected by the in-vehicle sensors of the vehicle and the roadside equipment and the status information of the target vehicle to construct a global environmental model.

[0063] In scenarios without satellite positioning signals, especially in roadways or tunnels, due to factors such as complex environments, low visibility, and mixed traffic of people and vehicles, rear-end collisions and multi-vehicle chain collisions often occur, resulting in serious casualties and losses. Through multi-sensor fusion (such as vision, lidar, millimeter-wave radar, etc.) and V2V communication technology, the perception accuracy and reliability of the operating vehicle for traffic information ahead can be improved.

[0064] In this embodiment, in-vehicle sensors of the vehicle itself such as cameras, lidar, and millimeter-wave radar can be used to perceive the surrounding environment and identify targets such as obstacles, pedestrians, and vehicles. Among them, the camera can identify lane lines, obstacles, pedestrians, traffic signs, etc. The lidar (LiDAR) can generate high-precision point cloud data for obstacle detection and mapping. The millimeter-wave radar can detect targets at a long distance and is suitable for harsh environments such as low light and water mist.

[0065] The environmental perception data collected by roadside equipment can include the perception information obtained by comprehensively perceiving the road traffic conditions through sensors such as cameras and lidar deployed on the roadside, and the perception information can be broadcast to surrounding vehicles through technologies such as CV-V2X. C-V2X technology is a vehicle-to-everything communication technology based on cellular networks and has the following advantages: high bandwidth and low latency: It can support the real-time transmission of a large amount of data and meet the communication requirements of intelligent driving. Wide coverage: The cellular network has a wide coverage range and can support vehicle communication in various scenarios. High reliability: The cellular network has high reliability and can ensure the stability of communication.

[0066] See Figure 2 , Figure 2 shows the process of fusing the vehicle's own positioning technology and environmental perception information. When performing perception information fusion, preprocessing (such as noise filtering, data alignment, etc.) can be carried out first, and then matching can be performed through methods such as multi-frame data association to track the movement trajectories of dynamic targets (such as vehicles and pedestrians) to ensure target consistency. In addition, algorithms such as Kalman filtering and particle filtering can be used to fuse the vehicle's own perception information, the perception information provided by the RSU, and the information transmitted by the target vehicle to improve the perception accuracy, build a global environment model, and can also generate a high-precision map, including static environments (such as road structures) and dynamic environments (such as vehicles and pedestrians).

[0067] Step 105, perform control decisions based on the second positioning information and the global environment model to achieve safe distance maintenance and collision avoidance.

[0068] In this embodiment, achieving safe distance keeping and collision avoidance can be to achieve a collision warning response of less than 0.5 seconds and a distance control accuracy of less than 2 meters. The specific values can be selected according to actual needs. The control decisions can include decision-making contents such as path planning, obstacle avoidance, speed control, and cooperative driving. Among them, path planning can include using algorithms such as Dijkstra to perform path planning based on high-precision positioning and global environment models, and selecting the optimal driving path. Obstacle avoidance can include using collision avoidance algorithms to avoid obstacles according to the perceived obstacle information to ensure safe driving. Speed control can include using algorithms such as proportional-integral-derivative control (PID) and model predictive control (MPC) to perform speed control according to traffic conditions and its own state to maintain a safe distance. Cooperative driving can include information interaction with other vehicles through C-V2X technology to achieve cooperative driving and improve traffic efficiency and safety.

[0069] Among them, the output formula of PID control can be as follows:

[0070] ;

[0071] Among them, u(t) represents the control output, such as acceleration or deceleration, and e(t) represents the error (such as the distance error from the target vehicle, speed error, etc.). represents the error at any moment of, and is used to calculate the cumulative sum of the errors from the start of the control process to the current time t, aiming to eliminate the steady-state error of the system. K p , K i , K d respectively represent the proportional gain, integral gain, and derivative gain.

[0072] MPC minimizes the objective function by optimizing the control input within a future period of time:

[0073] ;

[0074] Among them, x k represents the system state, x ref represents the reference state, u k represents the control input, and Q and R represent the weight matrices.

[0075] In addition, when it is detected that the target vehicle has an emergency braking or deceleration behavior, an alarm signal can be sent to the driver. According to the behavior prediction result of the target vehicle, the speed of the vehicle can be automatically adjusted to maintain a safe distance, and when the collision risk is relatively high, emergency braking can be triggered to avoid accidents. Among them, when the predicted time to collision is less than the threshold and the braking distance is greater than the current distance, emergency braking can be triggered. The behavior prediction result of the target vehicle can include determining whether the target vehicle is in an emergency braking or deceleration state through the motion state information of the target vehicle (such as acceleration, braking signal, etc.). For example, it can be determined by comparing the acceleration of the target vehicle with a pre-set acceleration threshold.

[0076] As an example, the alarm trigger condition can be judged by the following formula:

[0077] ;

[0078] where d SV-TV represents the distance between the vehicle itself and the target vehicle, d safe represents the safe distance, and the formula is as follows:

[0079] ;

[0080] where v SV represents the speed of the vehicle itself, v TV represents the speed of the target vehicle, t react represents the driver's reaction time (usually 1 - 2 seconds), and a max represents the maximum deceleration of the vehicle itself.

[0081] The vehicle control decision-making method based on the perception fusion technology provided by the present invention proposes a new intelligent driving solution, which can fuse the data of the vehicle's own sensors with the data obtained by other positioning and perception technologies in an environment without satellite positioning signals, ensuring that the vehicle can react in time when the target vehicle has a gentle stop or emergency braking to avoid collisions. By reducing the accident rate and the impact of the harsh environment on production efficiency, the safety and efficiency of the operation vehicle in the scenario without satellite positioning signals can be improved.

[0082] In some optional implementation manners, the roadside auxiliary positioning technology includes at least one of the following: ultra-wideband positioning (the positioning accuracy can be better than 0.1 meter), geomagnetic matching positioning (adapting to the metal interference environment), and radio signal fingerprint positioning (covering complex structure areas); the vehicle-mounted sensors include cameras, lidars, and millimeter-wave radars, and the roadside devices include cameras and lidars.

[0083] In some alternative implementation manners, the first positioning information of the vehicle is obtained through data fusion of at least two complementary sensors, including: fusing the data of at least two complementary sensors through a Kalman filter or an extended Kalman filter algorithm to eliminate noise and improve the positioning accuracy. The formula of the Kalman filter is as follows:

[0084] ;

[0085] ;

[0086] wherein, represents the prior state estimate at the k-th moment (i.e., the estimate before receiving the observation data at the k-th moment), A represents the state transition matrix, represents the state estimate value at the (k - 1)-th moment, B represents the control input matrix, represents the control input at the (k - 1)-th moment, represents the prior error covariance matrix at the k-th moment, represents the posterior error covariance matrix at the (k - 1)-th moment, and Q represents the process noise covariance matrix.

[0087] The state prediction formula is as follows:

[0088] ;

[0089] ;

[0090] ;

[0091] wherein, K k represents the Kalman gain, H represents the observation matrix, F represents the observation noise covariance matrix, z k represents the observation value at the k-th moment, represents the posterior state estimate value at the k-th moment, and P k represents the state prediction covariance matrix at the k-th moment.

[0092] In some alternative implementation manners, fusing the environmental perception data collected by the in-vehicle sensors and roadside devices of the vehicle and the state information of the target vehicle, including: adopting a target tracking algorithm to match the target vehicle perceived by the vehicle with the state information of the target vehicle, and implementing data fusion through a Kalman filter or a particle filter algorithm. Common target matching algorithms include the Nearest Neighbor algorithm and the Hungarian Algorithm, etc.

[0093] wherein, the cost function of the target matching of the Nearest Neighbor algorithm can be expressed as:

[0094] ;

[0095] Among them, p i SV represents the position of the i-th target sensed by the vehicle itself, and p j TV represents the position of the j-th target transmitted by the target vehicle through V2V.

[0096] The Hungarian algorithm realizes target matching by minimizing the total matching cost:

[0097] ;

[0098] Among them, C ij represents the matching cost, x ij represents the matching indication variable. When x ij = 1, it means a match; otherwise, it is 0. N represents the total number of targets sensed by the vehicle itself, and M represents the number of targets transmitted by the target vehicle.

[0099] The purpose of multi-sensor data fusion is to fuse the data of the vehicle's own sensors, the information transmitted by the target vehicle, and the environmental information provided by the RSU to construct a global environmental model. Commonly used fusion algorithms include Kalman filtering and extended Kalman filtering.

[0100] In some alternative implementation manners, the global environmental model can be a set of dynamic target states , where p i is the three-dimensional position coordinate of the i-th target, v i is the velocity vector of the i-th target, a i is the acceleration vector of the i-th target, and N is the total number of targets sensed by the vehicle itself. The target state is updated through a spatio-temporal synchronization algorithm.

[0101] In some alternative implementation manners, control decisions are made based on the second positioning information and the global environmental model, including evaluating the collision risk based on the following formula:

[0102] ;

[0103] ;

[0104] Among them, t collision represents the predicted time to collision, d stop represents the braking distance of the vehicle itself, d SV-TV represents the relative distance between the vehicle itself and the target vehicle, v TV represents the speed of the target vehicle, t react represents the driver's or system's reaction time, and the typical value can be 0.5 - 1 second, v SV represents the speed of the vehicle itself, amax represents the maximum acceleration of the vehicle; when the following formula is satisfied, emergency braking is triggered:

[0105] ;

[0106] ;

[0107] where t thresh represents a pre-set collision time threshold, with a value range of 1 - 3 seconds, which can be dynamically adjusted according to the scenario.

[0108] In some alternative implementation methods, the first positioning information of the vehicle is obtained through data fusion of at least two complementary sensors, including: adopting a multi-sensor fusion positioning algorithm based on federated learning, where at least two sensors independently train local positioning models, and aggregate global model parameters through a federated learning framework. The aggregation weight of federated learning is determined according to the following formula:

[0109] ;

[0110] where n represents the number of sensors, and W k represents the local positioning model parameters of the k-th sensor. In the federated learning framework, each sensor (such as an inertial navigation system, visual odometer, lidar odometer) independently trains and generates local model parameters based on its own collected positioning data and environmental information. α k represents the weight proportion of the k-th sensor in the aggregation of the federated learning model after normalization processing, and the calculation formula is as follows:

[0111] ;

[0112] where C i is the confidence score of the i-th sensor, and C k is the confidence score of the k-th sensor, which is calculated based on historical errors and current environmental parameters, and its calculation formula is:

[0113] ;

[0114] where α and β are adjustment coefficients, which are dynamically adjusted according to the real-time scenario and satisfy that the sum of α and β is 1, and σ k 2 is the historical positioning error variance of the k-th sensor, which is calculated through sliding window statistics, and E k is the environmental parameter score, and the calculation formula is:

[0115] ;

[0116] where D k represents the dust concentration of the current scenario, and Lk is the light intensity, is the water mist density, γ 1 , γ 2 , γ 3 is the preset environment weight coefficient, and satisfies γ 1 +γ 2 +γ 3 = 1. In addition, the fusion weights of each sensor can also be dynamically adjusted based on algorithms such as Q - learning to adapt to the noise characteristics in different environments.

[0117] In some alternative implementation manners, the method further includes: simulating the sensor noise in extreme environments through a generative adversarial network, training a robust fusion model, and detecting abnormal patterns in the input data in real time. If adversarial interference is detected, a redundant sensor data replacement mechanism is activated.

[0118] In some alternative implementation manners, making a control decision based on the second positioning information and the global environment model includes:

[0119] Calculating the collision probability through the following formula:

[0120] ;

[0121] where T represents the total number of time steps, d t represents the predicted distance deviation, μ t and σ t are the mean and variance of the spatio - temporal distribution. When P collision is greater than the preset probability value, emergency braking is triggered.

[0122] In addition, an LSTM network can also be used to predict the behavior of the target vehicle based on time - series data, dynamically update the collision risk threshold based on Bayesian probability to adapt to the uncertainty of driver behavior. Sensor noise in extreme environments is simulated through algorithms such as generative adversarial network (GAN), a robust fusion model is trained, and abnormal patterns in the input data are detected in real time. If adversarial interference is detected, redundant sensor data replacement is activated.

[0123] The vehicle control decision - making device provided by the present invention is described below. The vehicle control decision - making device based on perception fusion technology described below can be correspondingly referred to the vehicle control decision - making method based on perception fusion technology described above.

[0124] Figure 3 is the structural schematic diagram of the vehicle control decision - making device based on perception fusion technology provided by the embodiment of the present application, as Figure 3As shown in the figure, it specifically includes: a first acquisition module 301 configured to obtain the first positioning information of the vehicle through data fusion of at least two complementary sensors, where the complementary sensors include: an inertial navigation system, a visual odometer, a lidar odometer, or an odometer; a calibration module 302 configured to calibrate the first positioning information through a roadside assisted positioning technology to obtain the second positioning information; a second acquisition module 303 configured to obtain the status information of the target vehicle through a cellular vehicle-to-everything (C-V2X) communication technology; a fusion module 304 configured to fuse the environmental perception data collected by the vehicle's on-board sensors and roadside devices and the status information of the target vehicle to construct a global environmental model; and a control module 305 configured to make a control decision based on the second positioning information and the global environmental model to achieve safe distance keeping and collision avoidance.

[0125] The vehicle control decision-making device based on the perception fusion technology provided by the present invention can fuse the data of the vehicle's own sensors with the data obtained by other positioning and perception technologies in an environment without satellite positioning signals, improving the vehicle's perception, decision-making, and control capabilities.

[0126] Figure 4 An example of the physical structure diagram of an electronic device is shown as Figure 4 As shown in the figure, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440. Among them, the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call the logical instructions in the memory 430 to execute the vehicle control decision-making method based on the perception fusion technology. The method includes: obtaining the first positioning information of the vehicle through data fusion of at least two complementary sensors, where the complementary sensors include: an inertial navigation system, a visual odometer, a lidar odometer, or an odometer; calibrating the first positioning information through a roadside assisted positioning technology to obtain the second positioning information; obtaining the status information of the target vehicle through a cellular vehicle-to-everything (C-V2X) communication technology; fusing the environmental perception data collected by the vehicle's on-board sensors and roadside devices and the status information of the target vehicle to construct a global environmental model; and making a control decision based on the second positioning information and the global environmental model to achieve safe distance keeping and collision avoidance.

[0127] In addition, when the logical instructions in the above-mentioned memory 430 can be implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0128] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the vehicle control decision-making method based on the perception fusion technology provided by the above-mentioned various methods. The method includes: obtaining the first positioning information of the vehicle through data fusion of at least two complementary sensors, where the complementary sensors include: an inertial navigation system, a visual odometer, a lidar odometer, or an odometer; correcting the first positioning information through roadside auxiliary positioning technology to obtain the second positioning information; obtaining the status information of the target vehicle through cellular vehicle-to-everything communication technology; fusing the environmental perception data collected by the vehicle-mounted sensors of the vehicle and roadside devices and the status information of the target vehicle to construct a global environmental model; and making a control decision based on the second positioning information and the global environmental model to achieve safe distance keeping and collision avoidance.

[0129] In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the vehicle control decision-making method based on the perception fusion technology provided by the above-mentioned various methods. The method includes: obtaining the first positioning information of the vehicle through data fusion of at least two complementary sensors, where the complementary sensors include: an inertial navigation system, a visual odometer, a lidar odometer, or an odometer; correcting the first positioning information through roadside auxiliary positioning technology to obtain the second positioning information; obtaining the status information of the target vehicle through cellular vehicle-to-everything communication technology; fusing the environmental perception data collected by the vehicle-mounted sensors of the vehicle and roadside devices and the status information of the target vehicle to construct a global environmental model; and making a control decision based on the second positioning information and the global environmental model to achieve safe distance keeping and collision avoidance.

[0130] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0131] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vehicle control decision method based on perception fusion technology, characterized in that: include: Acquire first positioning information of the vehicle by data fusion of at least two complementary sensors, wherein the complementary sensors include: an inertial navigation system, a visual odometer, a laser radar odometer or an odometer; Correcting the first positioning information by using a roadside auxiliary positioning technology to obtain second positioning information; Obtaining the status information of the target vehicle through cellular vehicle networking communication technology; The global environment model is constructed by integrating the environmental perception data collected by the vehicle's onboard sensors and roadside equipment and the status information of the target vehicle; A control decision is made based on the second positioning information and the global environment model to achieve safe vehicle distance maintenance and collision avoidance.

2. The method according to claim 1, characterized in that The roadside auxiliary positioning technology includes at least one of the following: ultra-wideband positioning, geomagnetic matching positioning and radio signal fingerprint positioning; the vehicle-mounted sensors include cameras, lidars and millimeter-wave radars, and the roadside equipment includes cameras and lidars.

3. The method according to claim 1, characterized in that The obtaining of the first positioning information of the vehicle by fusing data of at least two complementary sensors includes: The data of at least two complementary sensors are fused through Kalman filtering or extended Kalman filtering algorithm to eliminate noise and improve positioning accuracy.

4. The method according to claim 1, characterized in that: The fusion of the environmental perception data collected by the vehicle-mounted sensors and the roadside equipment and the state information of the target vehicle includes: The target tracking algorithm is used to match the target vehicle perceived by the vehicle with the state information of the target vehicle, and the data fusion is achieved through the Kalman filter or particle filter algorithm.

5. The method according to claim 1, characterized in that: The global environment model is a dynamic target state set , where p i is the three-dimensional position coordinate of the i-th target, v i is the velocity vector of the i-th target, a i is the acceleration vector of the i-th target, N is the total number of targets sensed by the vehicle, and the target state is updated through the spatiotemporal synchronization algorithm.

6. The method according to claim 1, characterized in that The making a control decision based on the second positioning information and the global environment model includes: The collision risk is assessed based on the following formula: ； ； Among them, t collision represents the estimated time of collision, d stop Represents the braking distance of the vehicle, d SV-TV Represents the relative distance between the vehicle and the target vehicle, v TV represents the speed of the target vehicle, t react represents the driver or system reaction time, v SV represents the speed of the vehicle, a max Represents the maximum acceleration of the vehicle; Emergency braking is triggered when the following formula is met: ； ； Among them, t thresh Represents the preset collision time threshold, which ranges from 1 to 3 seconds and is adjusted dynamically according to the scenario.

7. The method according to claim 1, characterized in that The obtaining of the first positioning information of the vehicle by fusing data of at least two complementary sensors includes: A multi-sensor fusion positioning algorithm based on federated learning is adopted. The at least two complementary sensors independently train the local positioning model, and the global model parameters are aggregated through the federated learning framework. The aggregation weight of the federated learning is determined according to the following formula: ； Where n represents the number of sensors, W k represents the local positioning model parameters of the kth sensor, α k It represents the weight ratio of the kth sensor in the federated learning model aggregation after normalization. The calculation formula is as follows: ； Among them, C i is the confidence score of the i-th sensor, C k is the confidence score of the kth sensor, which is calculated based on historical errors and current environmental parameters. The calculation formula is: ； Among them, α and β are adjustment coefficients, which are dynamically adjusted according to the real-time scene, and the sum of α and β is 1, σ k 2 is the historical positioning error variance of the kth sensor, calculated by sliding window statistics, E k is the environmental parameter score, and the calculation formula is: ； Among them, D k Indicates the dust concentration in the current scene, L k is the light intensity, is the water mist density, γ1, γ2, γ3 are the preset environmental weight coefficients, and γ1+γ2+γ3=1.

8. The method according to claim 1, characterized in that: The method further comprises: By generating adversarial networks to simulate sensor noise in extreme environments, the robust fusion model is trained and abnormal patterns in the input data are detected in real time. If adversarial interference is detected, the redundant sensor data replacement mechanism is activated.

9. A vehicle control decision device based on perception fusion technology, characterized in that: include: A first acquisition module is configured to acquire first positioning information of the vehicle by data fusion of at least two complementary sensors, wherein the complementary sensors include: an inertial navigation system, a visual odometer, a laser radar odometer or an odometer; a correction module, configured to correct the first positioning information by using a roadside auxiliary positioning technology to obtain second positioning information; A second acquisition module is configured to acquire the state information of the target vehicle through cellular vehicle networking communication technology; A fusion module is configured to fuse the environmental perception data collected by the vehicle-mounted sensor and the roadside equipment and the state information of the target vehicle to build a global environmental model; The control module is configured to make control decisions based on the second positioning information and the global environment model to achieve safe vehicle distance maintenance and collision avoidance.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the vehicle control decision method based on perception fusion technology as described in any one of claims 1 to 8 is implemented.

Images