Intelligent vehicle obstacle sensing and warning method based on millimeter wave radar

Abstract

The application discloses a millimeter wave radar-based intelligent vehicle obstacle sensing early warning method, and belongs to the technical field of vehicle obstacle sensing, which comprises the following steps: collecting intelligent vehicle environment perception multi-modal data and processing the same to determine intelligent vehicle environment perception fusion data; analyzing the intelligent vehicle environment perception fusion data, predicting intelligent vehicle obstacle sensing risk, and determining intelligent vehicle obstacle sensing risk prediction results; and according to the intelligent vehicle obstacle sensing risk prediction results, corresponding risk early warning and control are performed on the intelligent vehicle obstacle sensing to ensure the driving safety of the intelligent vehicle. The application solves the problem that the existing method cannot realize comprehensive environment perception, the obstacle sensing detection effect is poor, collision early warning control cannot be effectively realized, and the driving safety of the intelligent vehicle cannot be ensured. The application can realize comprehensive environment perception, improve the obstacle sensing detection effect and precision, effectively realize collision early warning control, and ensure the driving safety of the intelligent vehicle.

Description

Technical Field

[0001] This invention relates to the field of vehicle obstacle sensing technology, specifically to an intelligent vehicle obstacle sensing and early warning method based on millimeter-wave radar. Background Technology

[0002] Intelligent vehicle obstacle sensing accurately identifies and tracks obstacles (such as pedestrians, vehicles, road signs, etc.) on the road under various complex road conditions, thereby providing a reliable basis for vehicle path planning and obstacle avoidance decisions.

[0003] Existing obstacle sensing technology for intelligent vehicles suffers from poor obstacle detection performance due to the inability of a single sensor to meet the perception requirements in complex environments. This results in a lack of comprehensive environmental perception, hindering effective collision warning and control, and ultimately failing to ensure the driving safety of intelligent vehicles. Summary of the Invention

[0004] The purpose of this invention is to provide an obstacle sensing and early warning method for intelligent vehicles based on millimeter-wave radar, which can achieve comprehensive environmental perception, improve the effect and accuracy of obstacle sensing and detection, effectively realize collision early warning and control, ensure the driving safety of intelligent vehicles, and solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] Intelligent vehicle obstacle sensing and early warning methods based on millimeter-wave radar include:

[0007] Collect and process multimodal environmental perception data of intelligent vehicles, extract features and perform weighted fusion to determine the fused environmental perception data of intelligent vehicles;

[0008] Analyze the environmental perception fusion data of intelligent vehicles to predict the obstacle sensing risk of intelligent vehicles and determine the prediction results of the obstacle sensing risk of intelligent vehicles.

[0009] Based on the risk prediction results of obstacle sensing for intelligent vehicles, corresponding risk warnings and controls are implemented to ensure the driving safety of intelligent vehicles.

[0010] Preferably, collect multimodal environmental perception data from intelligent vehicles and perform the following operations:

[0011] IoT devices are deployed on the front and rear bumpers and sides of smart vehicles, covering a 360° sensing range;

[0012] The intelligent vehicle uses millimeter-wave radar deployed on the front and rear bumpers and sides to monitor the surrounding environment in real time and obtain intelligent vehicle environmental perception millimeter-wave radar data.

[0013] High-definition cameras deployed on the front and rear bumpers and sides of intelligent vehicles are used to monitor the surrounding environment of intelligent vehicles in real time and obtain high-definition video data of intelligent vehicle environmental perception.

[0014] The LiDAR deployed on the front and rear bumpers and sides of the intelligent vehicle is used to monitor the surrounding environment of the intelligent vehicle in real time and obtain the environmental perception LiDAR data of the intelligent vehicle.

[0015] Based on the millimeter-wave radar data, high-definition camera data, and lidar data for intelligent vehicle environmental perception, multimodal data for intelligent vehicle environmental perception are determined.

[0016] Preferably, the multimodal data of the intelligent vehicle's environmental perception is processed by performing the following operations:

[0017] Clean the multimodal data of intelligent vehicle environmental perception, find and correct identifiable errors in the multimodal data of intelligent vehicle environmental perception, remove noise in the multimodal data of intelligent vehicle environmental perception, and reduce the interference of noise on the obstacle sensing of intelligent vehicle.

[0018] Normalize the multimodal data of intelligent vehicle environmental perception by transforming dimensional expressions into dimensionless expressions, thereby removing dimensional differences in the multimodal data of intelligent vehicle environmental perception and forming standardized multimodal data of intelligent vehicle environmental perception.

[0019] Preferably, the multimodal data of the intelligent vehicle's environmental perception is processed by performing the following operations:

[0020] The system integrates multimodal data of intelligent vehicle environmental perception, combines multimodal data of intelligent vehicle environmental perception from different sources into a unified view, eliminates data silos and forms globally shared multimodal data of intelligent vehicle environmental perception, and loads the multimodal data of intelligent vehicle environmental perception into a repository.

[0021] Feature extraction is performed on the multimodal data of intelligent vehicle environmental perception. Features related to obstacle sensing of intelligent vehicles are extracted from the multimodal data of intelligent vehicle environmental perception, and the extracted features are weighted and fused to form fused data of intelligent vehicle environmental perception.

[0022] Preferably, the environmental perception fusion data of intelligent vehicles is analyzed to predict the obstacle sensing risk of intelligent vehicles, and the following operations are performed:

[0023] Based on the requirements for obstacle sensing and early warning of intelligent vehicles, an obstacle sensing risk prediction model for intelligent vehicles is constructed and deployed.

[0024] The intelligent vehicle environmental perception fusion data is input into the intelligent vehicle obstacle sensing risk prediction model. The intelligent vehicle obstacle sensing risk prediction model analyzes and identifies the intelligent vehicle environmental perception fusion data, automatically predicts the intelligent vehicle obstacle sensing risk, and determines the intelligent vehicle obstacle sensing risk prediction result.

[0025] Preferably, an intelligent vehicle obstacle sensing risk prediction model is constructed, and the following operations are performed:

[0026] Collect historical obstacle sensing data of intelligent vehicles, and divide the collected historical obstacle sensing data of intelligent vehicles into training set and test set.

[0027] Based on deep learning technology, a training set is used to train the deep learning model, enabling the deep learning model to autonomously learn the obstacle sensing risk prediction behavior of intelligent vehicles from the training set and automatically predict the obstacle sensing risk of intelligent vehicles, thereby determining the obstacle sensing risk prediction model of intelligent vehicles.

[0028] The intelligent vehicle obstacle sensing risk prediction model is tested and evaluated based on the test set. The model test evaluation results are determined, and the parameters of the intelligent vehicle obstacle sensing risk prediction model are adjusted and optimized based on the model test evaluation results, so as to determine the optimal intelligent vehicle obstacle sensing risk prediction model.

[0029] Preferably, the intelligent vehicle obstacle sensing risk prediction model is tested and evaluated based on the test set, and the following operations are performed:

[0030] The test set is input into the intelligent vehicle obstacle sensing risk prediction model. The intelligent vehicle obstacle sensing risk prediction model is tested based on the test set, and its performance is evaluated to determine whether the intelligent vehicle obstacle sensing risk prediction model can achieve the expected effect of automatically predicting intelligent vehicle obstacle sensing risks.

[0031] When the intelligent vehicle obstacle sensing risk prediction model can achieve the expected effect of automatically predicting the intelligent vehicle obstacle sensing risk, the optimal intelligent vehicle obstacle sensing risk prediction model is directly determined.

[0032] When the intelligent vehicle obstacle sensing risk prediction model fails to achieve the expected effect of automatically predicting intelligent vehicle obstacle sensing risks, the parameters of the intelligent vehicle obstacle sensing risk prediction model are adjusted and optimized until the intelligent vehicle obstacle sensing risk prediction model can achieve the expected effect of automatically predicting intelligent vehicle obstacle sensing risks, thereby determining the optimal intelligent vehicle obstacle sensing risk prediction model.

[0033] Preferably, based on the obstacle sensing risk prediction results of intelligent vehicles, corresponding risk warnings and controls are implemented for obstacle sensing of intelligent vehicles, and the following operations are performed:

[0034] Based on the risk prediction results of obstacle sensing for intelligent vehicles, the risk level of obstacle sensing for intelligent vehicles is classified into low risk, medium risk, or high risk; and corresponding risk warnings and controls are implemented according to different risk levels.

[0035] When the risk is low, the dashboard will issue a warning and automatically indicate that there is a stationary obstacle 50 meters ahead;

[0036] In cases of medium risk, an automatic deceleration system will issue a warning, automatically reducing the speed to 10-20 km / h;

[0037] In high-risk situations, an emergency braking warning is issued, automatically triggering the automatic emergency braking system.

[0038] Preferably, the intelligent vehicle obstacle sensing and warning method based on millimeter-wave radar further includes:

[0039] Obtain the real-time location of intelligent vehicles;

[0040] Obtain the environmental perception history of other intelligent vehicles within a preset range around the real-time location within the most recent preset first time period.

[0041] For each piece of data in the history of environmental perception, obtain a value score for the data's value in the current or future obstacle sensing and warning of the intelligent vehicle. When the value score exceeds the score threshold, the data is included in the collected intelligent vehicle environmental perception multimodal data.

[0042] The steps for obtaining the value score include:

[0043] To obtain the current and future driving status of intelligent vehicles;

[0044] Based on current and future driving conditions, determine the set of value data requirements for intelligent vehicles;

[0045] Determine the maximum degree of conformity between the data and the value data requirements set;

[0046] Assess the credibility of the data;

[0047] Based on maximum relevance and credibility, the value score is calculated using the following formula:

[0048] val = (D1·F + D2·K) δ

[0049] Where val is the value score, F is the maximum compliance, K is the credibility, D1 is the preset weight corresponding to the maximum compliance, D2 is the preset weight corresponding to the credibility, and δ is the preset error coefficient.

[0050] Preferably, when performing corresponding risk warnings and controls on obstacle sensing of intelligent vehicles based on the risk prediction results of obstacle sensing of intelligent vehicles, the following operations are also performed:

[0051] Acquire the current driving status of the driver of the intelligent vehicle and the predicted road conditions for the intelligent vehicle in the next preset second time period;

[0052] Based on the current driving status and predicted road conditions, determine the upper limit of the driver's current cognitive ability to receive the obstacle sensing risk prediction results of the intelligent vehicle.

[0053] Based on the upper limit of cognitive ability, the best risk warning information generation template is matched from the risk warning information generation template library;

[0054] Based on the best risk warning information generation template, risk warning information is generated according to the obstacle sensing risk prediction results of intelligent vehicles;

[0055] Providing risk warning information to drivers of intelligent vehicles;

[0056] Receiving and responding to risk warning information;

[0057] Based on response information, identify early warning gaps in risk warning information;

[0058] Strengthen risk warning information to fill the warning gap;

[0059] Enhanced risk warning information is then sent to the driver of the intelligent vehicle again.

[0060] Compared with the prior art, the beneficial effects of the present invention are:

[0061] This invention combines millimeter-wave radar, high-definition cameras, and lidar to collect multimodal environmental perception data of intelligent vehicles. It processes this data, extracts features, and performs weighted fusion to determine the fused environmental perception data. By constructing an obstacle sensing risk prediction model for intelligent vehicles, the invention analyzes the fused environmental perception data to predict obstacle sensing risks. Based on the predicted risk results, corresponding risk warnings and controls are implemented for obstacle sensing, enabling comprehensive environmental perception, improving obstacle sensing detection effectiveness and accuracy, effectively achieving collision warning and control, and ensuring the driving safety of intelligent vehicles. Attached Figure Description

[0062] Figure 1 This is a flowchart of the intelligent vehicle obstacle sensing and early warning method of the present invention. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] To address the current limitations in achieving comprehensive environmental perception, resulting in poor obstacle detection, ineffective collision warning and control, and a failure to ensure the safety of intelligent vehicles, please refer to [link / reference needed]. Figure 1 This embodiment provides the following technical solution:

[0065] Intelligent vehicle obstacle sensing and early warning methods based on millimeter-wave radar include:

[0066] Collect and process multimodal environmental perception data of intelligent vehicles, extract features and perform weighted fusion to determine the fused environmental perception data of intelligent vehicles.

[0067] In this embodiment, multimodal data of environmental perception from intelligent vehicles are collected, and the following operations are performed:

[0068] IoT devices are deployed on the front and rear bumpers and sides of smart vehicles, covering a 360° sensing range;

[0069] The intelligent vehicle uses millimeter-wave radar deployed on the front and rear bumpers and sides to monitor the surrounding environment in real time and obtain intelligent vehicle environmental perception millimeter-wave radar data.

[0070] It should be noted that the data collected by millimeter-wave radar includes:

[0071] Distance: The distance between the target and the radar is calculated by measuring the time difference between the transmitted signal and the reflected signal.

[0072] Velocity: Utilizing the Doppler effect, the radial velocity of the target relative to the radar is detected.

[0073] Angle: The direction and angle of the target are determined by beam scanning or array antenna technology.

[0074] Reflection characteristics: Analyze the electromagnetic wave reflection characteristics of the target to distinguish different materials such as metal and plastic.

[0075] The data collected by the millimeter-wave radar is shown in Table 1:

[0076] Table 1: Millimeter-wave radar data for environmental perception of intelligent vehicles

[0077] Distance (m) Speed ​​(km / h) Angle (°) Reflective properties Target A 100 30 45 Metal Target B 80 20 30 plastic Target C 120 26 20 plastic Target D 150 40 50 Metal

[0078] High-definition cameras deployed on the front and rear bumpers and sides of intelligent vehicles are used to monitor the surrounding environment of intelligent vehicles in real time and obtain high-definition video data of intelligent vehicle environmental perception.

[0079] It should be noted that the data collected by the high-definition camera includes:

[0080] Image data: Captures two-dimensional visual information of the target, including color, texture, shape, etc.

[0081] Depth information: Obtaining distance information of the target through binocular vision or structured light technology.

[0082] Semantic information: Identify targets such as pedestrians, vehicles, and traffic signs through image recognition algorithms.

[0083] The LiDAR deployed on the front and rear bumpers and sides of the intelligent vehicle is used to monitor the surrounding environment of the intelligent vehicle in real time and obtain the environmental perception LiDAR data of the intelligent vehicle.

[0084] It should be noted that the data collected by the lidar includes:

[0085] 3D point cloud data: The target's position (x, y, z) and reflection intensity are obtained through laser scanning.

[0086] Distance: Laser ranging provides high-precision target distance information.

[0087] Speed: Some lidar systems can measure the speed of a target.

[0088] Reflection characteristics: Analyze the material and surface properties of the target.

[0089] Based on the millimeter-wave radar data, high-definition camera data, and lidar data for intelligent vehicle environmental perception, multimodal data for intelligent vehicle environmental perception are determined.

[0090] Therefore, by combining millimeter-wave radar, high-definition cameras, and lidar to collect multimodal data on the environment perception of intelligent vehicles, comprehensive environmental perception can be achieved, avoiding the difficulty of a single sensor meeting the perception needs in complex environments.

[0091] In this embodiment, the multimodal data of the intelligent vehicle's environmental perception is processed by performing the following operations:

[0092] Cleaning the multimodal environmental perception data of intelligent vehicles can identify and correct identifiable errors, remove noise from the data, and reduce noise interference with obstacle sensing, thereby improving the data quality of the multimodal environmental perception data of intelligent vehicles.

[0093] Normalization is performed on the multimodal data of intelligent vehicle environmental perception. Dimensional expressions are transformed into dimensionless expressions to remove dimensional differences in the multimodal data of intelligent vehicle environmental perception, forming standardized multimodal data of intelligent vehicle environmental perception, which is convenient for subsequent analysis of intelligent vehicle environmental perception multimodal data.

[0094] In this embodiment, the multimodal data of the intelligent vehicle's environmental perception is processed by performing the following operations:

[0095] The system integrates multimodal data of intelligent vehicle environmental perception, combines multimodal data of intelligent vehicle environmental perception from different sources into a unified view, eliminates data silos and forms globally shared multimodal data of intelligent vehicle environmental perception, and loads the multimodal data of intelligent vehicle environmental perception into a repository.

[0096] Feature extraction is performed on multimodal data of intelligent vehicle environmental perception. Features related to obstacle sensing of intelligent vehicles are extracted from the multimodal data of intelligent vehicle environmental perception, and the extracted features are weighted and fused to form intelligent vehicle environmental perception fusion data, thereby improving the accuracy and real-time performance of target detection.

[0097] Analyze the environmental perception fusion data of intelligent vehicles to predict the obstacle sensing risk of intelligent vehicles and determine the prediction results of the obstacle sensing risk of intelligent vehicles.

[0098] In this embodiment, the environmental perception fusion data of the intelligent vehicle is analyzed to predict the obstacle detection risk of the intelligent vehicle, and the following operations are performed:

[0099] Based on the requirements for obstacle sensing and early warning of intelligent vehicles, an obstacle sensing risk prediction model for intelligent vehicles is constructed and deployed.

[0100] This involves collecting historical data on obstacle sensing of intelligent vehicles, and then dividing the collected historical data into training and testing sets.

[0101] Based on deep learning technology, a training set is used to train the deep learning model, enabling the deep learning model to autonomously learn the obstacle sensing risk prediction behavior of intelligent vehicles from the training set and automatically predict the obstacle sensing risk of intelligent vehicles, thereby determining the obstacle sensing risk prediction model of intelligent vehicles.

[0102] The intelligent vehicle obstacle sensing risk prediction model is tested and evaluated based on the test set. The test set is input into the intelligent vehicle obstacle sensing risk prediction model, and the intelligent vehicle obstacle sensing risk prediction model is tested according to the test set. The performance of the intelligent vehicle obstacle sensing risk prediction model is evaluated, and it is determined whether the intelligent vehicle obstacle sensing risk prediction model can achieve the expected effect of automatically predicting the obstacle sensing risk of intelligent vehicles.

[0103] When the intelligent vehicle obstacle sensing risk prediction model can achieve the expected effect of automatically predicting the intelligent vehicle obstacle sensing risk, the optimal intelligent vehicle obstacle sensing risk prediction model is directly determined.

[0104] When the intelligent vehicle obstacle sensing risk prediction model fails to achieve the expected effect of automatically predicting the intelligent vehicle obstacle sensing risk, the parameters of the intelligent vehicle obstacle sensing risk prediction model are adjusted and optimized until the intelligent vehicle obstacle sensing risk prediction model can achieve the expected effect of automatically predicting the intelligent vehicle obstacle sensing risk, thereby determining the optimal intelligent vehicle obstacle sensing risk prediction model.

[0105] The intelligent vehicle environmental perception fusion data is input into the intelligent vehicle obstacle sensing risk prediction model. The intelligent vehicle obstacle sensing risk prediction model analyzes and identifies the intelligent vehicle environmental perception fusion data, automatically predicts the intelligent vehicle obstacle sensing risk, and determines the intelligent vehicle obstacle sensing risk prediction result.

[0106] Based on the risk prediction results of obstacle sensing for intelligent vehicles, corresponding risk warnings and controls are implemented to ensure the driving safety of intelligent vehicles.

[0107] In this embodiment, based on the obstacle sensing risk prediction results of the intelligent vehicle, corresponding risk warnings and controls are performed on the obstacle sensing of the intelligent vehicle, and the following operations are performed:

[0108] Based on the risk prediction results of obstacle sensing for intelligent vehicles, the risk level of obstacle sensing for intelligent vehicles is classified into low risk, medium risk, or high risk; and corresponding risk warnings and controls are implemented according to different risk levels.

[0109] When the risk is low, the dashboard will issue a warning and automatically indicate that there is a stationary obstacle 50 meters ahead;

[0110] In cases of medium risk, an automatic deceleration system will issue a warning, automatically reducing the speed to 10-20 km / h;

[0111] In high-risk situations, an emergency braking warning is issued, automatically triggering the automatic emergency braking system.

[0112] Specifically, based on the risk prediction results of obstacle sensing for intelligent vehicles, corresponding risk warnings and controls are implemented for obstacle sensing of intelligent vehicles. The risk warning and control measures for obstacle sensing of intelligent vehicles are shown in Table 2.

[0113] Table 2: Status of Obstacle Sensing Risk Warning and Control for Intelligent Vehicles

[0114] Risk level Risk warning and control Low risk The dashboard issues a warning, automatically indicating that there is a stationary obstacle 50 meters ahead. Medium risk The system automatically decelerates and issues a warning, reducing the speed to 10-20 km / h. High risk Emergency braking warning, automatically triggering the automatic emergency braking system.

[0115] In summary, by constructing an obstacle sensing risk prediction model for intelligent vehicles, analyzing the environmental perception fusion data of intelligent vehicles, predicting obstacle sensing risks, determining the prediction results, and conducting corresponding risk warnings and controls based on these results, comprehensive environmental perception can be achieved, the obstacle sensing detection effect and accuracy can be improved, collision warning and control can be effectively implemented, and the driving safety of intelligent vehicles can be ensured.

[0116] In this embodiment, the intelligent vehicle obstacle sensing and early warning method based on millimeter-wave radar further includes:

[0117] Obtain the real-time location of intelligent vehicles;

[0118] Obtain the environmental perception history of other intelligent vehicles within a preset range around the real-time location within the most recent preset first time period.

[0119] For each piece of data in the history of environmental perception, obtain a value score for the data's value in the current or future obstacle sensing and warning of the intelligent vehicle. When the value score exceeds the score threshold, the data is included in the collected intelligent vehicle environmental perception multimodal data.

[0120] The steps for obtaining the value score include:

[0121] To obtain the current and future driving status of intelligent vehicles;

[0122] Based on current and future driving conditions, determine the set of value data requirements for intelligent vehicles;

[0123] Determine the maximum degree of conformity between the data and the value data requirements set;

[0124] Assess the credibility of the data;

[0125] Based on maximum relevance and credibility, the value score is calculated using the following formula:

[0126] val = (D1·F + D2·K) δ

[0127] Where val is the value score, F is the maximum compliance, K is the credibility, D1 is the preset weight corresponding to the maximum compliance, D2 is the preset weight corresponding to the credibility, and δ is the preset error coefficient.

[0128] The working principle and beneficial effects of the above technical solution are as follows:

[0129] This method enhances the obstacle perception and warning capabilities of a smart vehicle by integrating its real-time location with historical environmental perception data from other smart vehicles in the vicinity. The implementation of this method first relies on acquiring the smart vehicle's real-time location, typically obtained through GPS or other positioning technologies. Real-time location refers to the vehicle's spatial coordinates at a specific moment, reflecting its current geographical environment. Next, the system collects historical environmental perception data from other smart vehicles within a preset range around the real-time location, covering a recent preset time period. The preset range is an area centered on the current vehicle with an adjustable radius (e.g., 100 meters), and the preset time period is a past time window, such as the last 5 minutes. This historical environmental perception data is collected by other smart vehicles using their onboard sensors (such as millimeter-wave radar, lidar, and cameras), and includes road conditions (e.g., road smoothness, potholes) and obstacle information (e.g., obstacle location, size, and speed). This data is inherently multi-source and multi-modal, reflecting the collective perception capabilities of surrounding vehicles.

[0130] After acquiring this historical data, the system needs to filter out the parts that are valuable for the current vehicle's obstacle sensing and warning capabilities, thus introducing a value scoring mechanism. Value scoring is an indicator used to quantify the potential contribution of each piece of historical data to the current vehicle. Its calculation process comprehensively considers two key factors: data relevance and reliability. Relevance refers to the degree to which historical data matches the current vehicle's needs, reflecting whether the data can meet the vehicle's information requirements in a specific scenario. Reliability is a measure of data reliability, typically related to the performance of the sensor from which the data originates and the freshness of the data acquisition time. To calculate relevance, the system first needs to acquire the intelligent vehicle's current and future driving conditions. Current driving conditions include the vehicle's real-time state parameters, such as speed (distance traveled per second), acceleration (rate of change of speed), and direction (vehicle's heading angle). Future driving conditions are the vehicle's possible states predicted based on the navigation path (preset driving route) or driving intentions (e.g., upcoming lane changes, turns, etc.). Based on this information, the system generates a set of valuable data requirements, which is a set of environmental data types that the vehicle may need in the current and future scenarios, such as the distance to obstacles ahead and the movement trajectory of vehicles to the side. For each piece of historical data, the system calculates its degree of conformity with each requirement in the value data requirement set, and selects the highest degree of conformity. The highest degree of conformity refers to the value with the highest matching degree among all requirements, reflecting the best applicability of the data. At the same time, the system also evaluates the credibility of the data, which may be quantified based on factors such as the reliability of the data source (e.g., high-precision sensors are better than low-precision sensors) and the timeliness of the data (the newer the data, the more reliable it is).

[0131] Ultimately, the value score is calculated using the formula val=(D1·F+D2·K) δThe formula is implemented as follows: val = (value score), F = maximum compliance, K = credibility, D1 = preset weight corresponding to the maximum compliance (used to adjust the importance of compliance in the score), D2 = preset weight corresponding to credibility (used to adjust the importance of credibility), and δ = preset error coefficient (used to correct non-linear effects or errors in the score). This formula balances the practicality and reliability of the data by weighted summation and applying an error coefficient. In intelligent vehicle environmental perception, the error coefficient δ typically ranges from 0 to 1, with the specific value designed according to the application scenario. For example, assuming maximum compliance F = 0.8, credibility K = 0.6, weights D1 = 0.7, D2 = 0.3, if δ = 0.5, then val = (0.7·0.8 + 0.3·0.6)^0.5 ≈ 0.86; if δ = 1.5, then val ≈ 0.79. When historical data contains noise or outliers, choosing δ (e.g., 0.5) can reduce the difference in high scores, improve the discrimination of low scores, and reduce the impact of outliers; if it is necessary to highlight high-value data, then choose δ > 1 (e.g., 1.5). The choice of δ is determined through experiments and model training. For example, cross-validation can be used to evaluate the impact of different δ values ​​on the screening accuracy, ultimately correcting nonlinear errors and improving data reliability.

[0132] When the value score of a piece of historical data exceeds a preset scoring threshold (a predefined critical value used to filter high-value data), that data will be incorporated into the current intelligent vehicle's environmental perception multimodal data. Environmental perception multimodal data refers to a collection of data from multiple sources and of multiple types, used to support the vehicle's obstacle perception and decision-making. This selected data will be combined with real-time data collected by the vehicle's own sensors (such as millimeter-wave radar) to form a more comprehensive view of environmental information.

[0133] By utilizing historical environmental perception data from surrounding vehicles, current vehicles can overcome the limitations of their own sensors' sensing range and capabilities to obtain richer and more accurate environmental information. While millimeter-wave radar has advantages in penetrating fog and detecting distant obstacles, its sensing range and resolution are limited. Historical data from other vehicles can supplement these shortcomings. For example, when the road ahead is obstructed or contains complex obstacles, the current vehicle may not be able to accurately determine the situation using only its own radar. However, by integrating the perception history of surrounding vehicles, potential risks can be identified in advance. This method significantly improves the accuracy (i.e., the probability of correctly identifying obstacles) and timeliness (i.e., the early warning window) of obstacle sensing warnings, thereby effectively reducing the risk of potential traffic accidents and providing important assurance for the safety of intelligent driving. Furthermore, the introduction of a value scoring mechanism ensures efficient data filtering, avoids interference from redundant or low-quality data, and improves the system's computational efficiency and practicality.

[0134] In this embodiment, when performing corresponding risk warnings and controls on intelligent vehicle obstacle sensing based on the intelligent vehicle obstacle sensing risk prediction results, the following operations are also performed:

[0135] Acquire the current driving status of the driver of the intelligent vehicle and the predicted road conditions for the intelligent vehicle in the next preset second time period;

[0136] Based on the current driving status and predicted road conditions, determine the upper limit of the driver's current cognitive ability to receive the obstacle sensing risk prediction results of the intelligent vehicle.

[0137] Based on the upper limit of cognitive ability, the best risk warning information generation template is matched from the risk warning information generation template library;

[0138] Based on the best risk warning information generation template, risk warning information is generated according to the obstacle sensing risk prediction results of intelligent vehicles;

[0139] Providing risk warning information to drivers of intelligent vehicles;

[0140] Receiving and responding to risk warning information;

[0141] Based on response information, identify early warning gaps in risk warning information;

[0142] Strengthen risk warning information to fill the warning gap;

[0143] Enhanced risk warning information is then sent to the driver of the intelligent vehicle again.

[0144] The working principle and beneficial effects of the above technical solution are as follows:

[0145] Based on the driver's current driving state and future road conditions, intelligent risk warning information is generated and dynamically adjusted to ensure the driver can effectively receive and respond to potential risks. The entire process begins with acquiring the driver's current driving state and predicted road conditions within a preset second time period. The current driving state refers to the driver's physiological and behavioral characteristics at a specific moment, such as attention level (determined by eye tracking to assess concentration), fatigue level (determined by facial expressions), and driving behavior (e.g., the smoothness of steering wheel operation). The preset second time period is an upcoming time window, such as the next 10 seconds or 1 minute. Predicted road condition information includes traffic flow (vehicle density on the road), road conditions (whether there is construction or damage), and weather changes (e.g., whether it will rain soon). This information is typically generated through vehicle-to-everything (V2X) technology or the vehicle's own predictive models.

[0146] Based on the driver's current driving state and predicted road conditions, the system assesses the driver's cognitive limit for receiving risk warning information. The cognitive limit refers to the upper limit of a driver's ability to effectively process and understand information in a specific situation, limited by current cognitive load (i.e., the burden on the brain to handle multiple tasks simultaneously). For example, the cognitive limit decreases when the driver is fatigued or distracted. Next, the system matches the optimal risk warning information generation template from a risk warning information generation template library. This library is a pre-set database containing various warning information formats and content designs, such as audio prompts ("Obstacle ahead"), image displays (icons on the dashboard), and vibration alerts (steering wheel vibration). Each template is suitable for different driving scenarios and driver states. The selection of the optimal template is based on the driver's cognitive limit, ensuring that the warning information conveys the necessary content without being overly complex or lengthy and exceeding the driver's processing capacity. Subsequently, based on the selected template and the intelligent vehicle obstacle sensing risk prediction results, the system generates specific risk warning information, such as "Stationary obstacle 50 meters ahead, please slow down."

[0147] After generating a risk warning, the system outputs this information to the driver through the vehicle's human-machine interface (such as speakers, displays, and haptic devices) and monitors the driver's reaction to the warning in real time. Reaction information refers to the driver's perception and behavioral feedback to the warning, such as changes in gaze (whether they look towards the warning area), operational actions (whether they slow down or steer), and voice feedback (whether they respond with "I understand"). Based on this reaction information, the system identifies warning gaps in the risk warning information—the parts where the warning information fails to achieve its intended effect. For example, if the driver does not slow down, it may be because they did not notice the warning or understand its urgency, thus constituting a warning gap. To fill this gap, the system reinforces the risk warning information. Reinforcement involves adjusting the presentation of the information (e.g., changing from a sound prompt to a sound with vibration), increasing the information intensity (increasing volume or frequency), and providing more detailed explanations (e.g., changing from "Obstacle ahead" to "Obstacle 50 meters ahead, please slow down immediately"). The reinforced risk warning information is then output to the driver again to ensure they fully understand and take appropriate action.

[0148] By dynamically adjusting the content and presentation of risk warning information, the system can adapt to the driver's current state and needs, significantly improving the effectiveness of the warning information (i.e., the probability of the driver correctly understanding and responding). Traditional warning systems often use fixed patterns, ignoring individual differences and real-time states of drivers, which may lead to poor information delivery or cognitive overload. This method, however, achieves personalization and closed-loop optimization by assessing cognitive limits and monitoring responses. Furthermore, this method can reduce risks caused by driver fatigue and inattention, especially in complex road conditions or severe weather, by timely reinforcing warning information to ensure the driver's perception and response to potential hazards. This continuously optimized warning strategy significantly improves driving safety (i.e., reduces the accident rate), providing strong support for the reliable operation of intelligent vehicles in real-world road environments, while also enhancing driver trust and user experience.

[0149] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0150] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for intelligent vehicle obstacle sensing and early warning based on millimeter-wave radar, characterized in that, include: The intelligent vehicle environmental perception fusion data is determined by collecting multimodal data of intelligent vehicle environment perception from millimeter-wave radar, processing the data, extracting features and performing weighted fusion. Among them, the environmental perception history is acquired. For each data point in the environmental perception history, a value score is obtained for the value of the data for the intelligent vehicle to perform obstacle sensing and warning in the present or future. When the value score exceeds the score threshold, the data is included in the collected intelligent vehicle environmental perception multimodal data. Analyze the environmental perception fusion data of intelligent vehicles to predict the obstacle sensing risk of intelligent vehicles and determine the prediction results of the obstacle sensing risk of intelligent vehicles. Based on the risk prediction results of obstacle sensing for intelligent vehicles, corresponding risk warnings and controls are implemented for obstacle sensing of intelligent vehicles to ensure the driving safety of intelligent vehicles. When performing risk warnings and controls on intelligent vehicle obstacle sensing based on the risk prediction results, the following operations are also performed: Acquire the current driving status of the driver of the intelligent vehicle and the predicted road conditions for the intelligent vehicle in the next preset second time period; Based on the current driving status and predicted road conditions, determine the upper limit of the driver's current cognitive ability to receive the obstacle sensing risk prediction results of the intelligent vehicle. Based on the upper limit of cognitive ability, the best risk warning information generation template is matched from the risk warning information generation template library; Based on the best risk warning information generation template, risk warning information is generated according to the obstacle sensing risk prediction results of intelligent vehicles; Providing risk warning information to drivers of intelligent vehicles; Receiving and responding to risk warning information; Based on response information, identify early warning gaps in risk warning information; Strengthen risk warning information to fill the warning gap; Enhanced risk warning information is then sent to the driver of the intelligent vehicle again.

2. The intelligent vehicle obstacle sensing and early warning method based on millimeter-wave radar according to claim 1, characterized in that, Also includes: Obtain the real-time location of intelligent vehicles; Obtain the environmental perception history of other intelligent vehicles within a preset range around the real-time location within the most recent preset first time period. For each piece of data in the history of environmental perception, obtain a value score for the data's value in the current or future obstacle sensing and warning of the intelligent vehicle. When the value score exceeds the score threshold, the data is included in the collected intelligent vehicle environmental perception multimodal data. The steps for obtaining the value score include: To obtain the current and future driving status of intelligent vehicles; Based on current and future driving conditions, determine the set of value data requirements for intelligent vehicles; Determine the maximum degree of conformity between the data and the value data requirements set; Assess the credibility of the data; Based on maximum relevance and credibility, the value score is calculated using the following formula: in, Rate the value. To achieve maximum compliance, For credibility, The preset weight corresponding to the maximum compliance. The preset weight corresponding to credibility. This is the preset error coefficient.

3. The intelligent vehicle obstacle sensing and early warning method based on millimeter-wave radar according to claim 2, characterized in that, The multimodal data of environmental perception from intelligent vehicles are processed, and the following operations are performed: The system integrates multimodal data of intelligent vehicle environmental perception, combines multimodal data of intelligent vehicle environmental perception from different sources into a unified view, eliminates data silos and forms globally shared multimodal data of intelligent vehicle environmental perception, and loads the multimodal data of intelligent vehicle environmental perception into a repository. Feature extraction is performed on the multimodal data of intelligent vehicle environmental perception. Features related to obstacle sensing of intelligent vehicles are extracted from the multimodal data of intelligent vehicle environmental perception, and the extracted features are weighted and fused to form intelligent vehicle environmental perception fusion data.

4. The intelligent vehicle obstacle sensing and early warning method based on millimeter-wave radar according to claim 3, characterized in that, Analyze the environmental perception fusion data of intelligent vehicles to predict obstacle detection risks and perform the following operations: Based on the requirements for obstacle sensing and early warning of intelligent vehicles, an obstacle sensing risk prediction model for intelligent vehicles is constructed and deployed. The intelligent vehicle environmental perception fusion data is input into the intelligent vehicle obstacle sensing risk prediction model. The intelligent vehicle obstacle sensing risk prediction model analyzes and identifies the intelligent vehicle environmental perception fusion data, automatically predicts the intelligent vehicle obstacle sensing risk, and determines the intelligent vehicle obstacle sensing risk prediction result.

5. The intelligent vehicle obstacle sensing and early warning method based on millimeter-wave radar according to claim 4, characterized in that, Collect multimodal environmental perception data from intelligent vehicles and perform the following operations: IoT devices are deployed on the front and rear bumpers and sides of smart vehicles, covering a 360° sensing range; The intelligent vehicle uses millimeter-wave radar deployed on the front and rear bumpers and sides to monitor the surrounding environment in real time and obtain intelligent vehicle environmental perception millimeter-wave radar data. High-definition cameras deployed on the front and rear bumpers and sides of intelligent vehicles are used to monitor the surrounding environment of intelligent vehicles in real time and obtain high-definition video data of intelligent vehicle environmental perception. The LiDAR deployed on the front and rear bumpers and sides of the intelligent vehicle is used to monitor the surrounding environment of the intelligent vehicle in real time and obtain the environmental perception LiDAR data of the intelligent vehicle. Based on the millimeter-wave radar data, high-definition camera data, and lidar data for intelligent vehicle environmental perception, multimodal data for intelligent vehicle environmental perception are determined.

6. The intelligent vehicle obstacle sensing and early warning method based on millimeter-wave radar according to claim 5, characterized in that, The multimodal data of environmental perception from intelligent vehicles are processed, and the following operations are performed: Clean the multimodal data of intelligent vehicle environmental perception, find and correct identifiable errors in the multimodal data of intelligent vehicle environmental perception, remove noise in the multimodal data of intelligent vehicle environmental perception, and reduce the interference of noise on the obstacle sensing of intelligent vehicle. The multimodal data of intelligent vehicle environmental perception is normalized by transforming dimensional expressions into dimensionless expressions, thereby removing dimensional differences in the multimodal data of intelligent vehicle environmental perception and forming standardized multimodal data of intelligent vehicle environmental perception.

7. The intelligent vehicle obstacle sensing and early warning method based on millimeter-wave radar according to claim 6, characterized in that, Construct an obstacle sensing risk prediction model for intelligent vehicles and perform the following operations: Collect historical obstacle sensing data of intelligent vehicles, and divide the collected historical obstacle sensing data of intelligent vehicles into training set and test set. Based on deep learning technology, a training set is used to train the deep learning model, enabling the deep learning model to autonomously learn the obstacle sensing risk prediction behavior of intelligent vehicles from the training set and automatically predict the obstacle sensing risk of intelligent vehicles, thereby determining the obstacle sensing risk prediction model of intelligent vehicles. The intelligent vehicle obstacle sensing risk prediction model is tested and evaluated based on the test set. The model test evaluation results are determined, and the parameters of the intelligent vehicle obstacle sensing risk prediction model are adjusted and optimized based on the model test evaluation results, so as to determine the optimal intelligent vehicle obstacle sensing risk prediction model.

8. The intelligent vehicle obstacle sensing and early warning method based on millimeter-wave radar according to claim 7, characterized in that, The intelligent vehicle obstacle sensing risk prediction model was tested and evaluated based on the test set, and the following operations were performed: The test set is input into the intelligent vehicle obstacle sensing risk prediction model. The intelligent vehicle obstacle sensing risk prediction model is tested based on the test set, and its performance is evaluated to determine whether the intelligent vehicle obstacle sensing risk prediction model can achieve the expected effect of automatically predicting intelligent vehicle obstacle sensing risks. When the intelligent vehicle obstacle sensing risk prediction model can achieve the expected effect of automatically predicting the intelligent vehicle obstacle sensing risk, the optimal intelligent vehicle obstacle sensing risk prediction model is directly determined. When the intelligent vehicle obstacle sensing risk prediction model fails to achieve the expected effect of automatically predicting intelligent vehicle obstacle sensing risks, the parameters of the intelligent vehicle obstacle sensing risk prediction model are adjusted and optimized until the intelligent vehicle obstacle sensing risk prediction model can achieve the expected effect of automatically predicting intelligent vehicle obstacle sensing risks, thereby determining the optimal intelligent vehicle obstacle sensing risk prediction model.

9. The intelligent vehicle obstacle sensing and early warning method based on millimeter-wave radar according to claim 8, characterized in that, Based on the risk prediction results of obstacle sensing for intelligent vehicles, corresponding risk warnings and controls are implemented, and the following operations are performed: Based on the risk prediction results of obstacle sensing for intelligent vehicles, the risk level of obstacle sensing for intelligent vehicles is classified into low risk, medium risk, or high risk; and corresponding risk warnings and controls are implemented according to different risk levels. When the risk is low, the dashboard will issue a warning and automatically indicate that there is a stationary obstacle 50 meters ahead; In cases of medium risk, an automatic deceleration system will issue a warning, automatically reducing the speed to 10-20 km / h; In high-risk situations, an emergency braking warning is issued, automatically triggering the automatic emergency braking system.

Images