Data model processing method and system

Abstract

The invention relates to the technical field of data processing, in particular to a data model processing method and system. The method comprises the following steps: acquiring multi-modal vehicle-mounted sensing data, and performing vehicle environment sensing to obtain dynamic vehicle environment sensing data; behavior pattern modeling is carried out through a vehicle-mounted sensing network, and a driver behavior prediction model is obtained; performing driving scene risk assessment based on the driver behavior prediction model and the dynamic vehicle environment perception data to obtain driving scene risk assessment data; performing vehicle multi-level early warning strategy analysis according to the driving scene risk assessment data to obtain a vehicle early warning instruction set; performing driver behavior prediction through a vehicle-mounted sensing network to obtain real-time early-warning driver behavior prediction data; and carrying out vehicle emergency intervention measure automatic decision making on the real-time driving early warning scene simulation data to obtain an emergency intervention decision execution instruction set. According to the invention, the driving safety and the response efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a data model processing method and system. Background Art

[0002] With the development of intelligent vehicle technology, the intelligence level of vehicles has been continuously improved. As a core component thereof, the in-vehicle monitoring system has become an important tool to ensure driving safety and enhance driving experience. In the field of vehicle monitoring, traditional vehicle monitoring methods mainly rely on the real-time collection and processing of in-vehicle sensor data, such as vehicle speed, engine status, brake usage, etc. Although these traditional monitoring systems can provide certain safety guarantees, with the increasing demand for intelligence, traditional methods show certain limitations in terms of real-time performance, data processing capabilities, and coping with complex driving scenarios.

[0003] Traditional methods usually monitor vehicle status based on preset rules and standards, lacking the adaptability to different driving environments and different drivers. For example, in different road environments or different weather conditions, the running status of the vehicle and the behavior performance of the driver will change greatly. Traditional methods are difficult to dynamically adjust monitoring strategies and behavior analysis models according to the actual situation, resulting in the system being slow or inaccurate in response. Summary of the Invention

[0004] Based on this, it is necessary for the present invention to provide a data model processing method and system to solve at least one of the above technical problems.

[0005] To achieve the above object, a data model processing method includes the following steps:

[0006] Step S1: Obtain multi-modal in-vehicle sensing data, and perform multi-modal sensing data fusion according to the multi-modal in-vehicle sensing data to obtain an in-vehicle sensing network; perform vehicle environment perception according to the in-vehicle sensing network to obtain dynamic vehicle environment perception data;

[0007] Step S2: Collect driver behavior data through the in-vehicle sensing network to obtain driver behavior data, and perform driver behavior pattern recognition according to the driver behavior data to obtain driver behavior pattern data; perform behavior pattern modeling on the driver behavior pattern data to obtain a driver behavior prediction model;

[0008] Step S3: Perform vehicle driving scenario simulation based on the driver behavior prediction model and the dynamic vehicle environment perception data to obtain vehicle driving scenario simulation data, and perform driving scenario risk assessment on the vehicle driving scenario simulation data to obtain driving scenario risk assessment data;

[0009] Step S4: Analyze the vehicle multi-level warning strategy based on the driving scenario risk assessment data to obtain a vehicle warning instruction set, and transmit it to the vehicle control platform to execute the vehicle driving warning task;

[0010] Step S5: Collect real-time driver behavior data and real-time vehicle environment perception through the in-vehicle sensor network to obtain real-time warning driver behavior data and real-time warning vehicle environment perception data, and perform driver behavior prediction on the real-time warning driver behavior data and real-time warning vehicle environment perception data through the driver behavior prediction model to obtain real-time warning driver behavior prediction data;

[0011] Step S6: Evaluate the vehicle warning feedback degree for the real-time driving warning scenario simulation data to obtain vehicle warning feedback degree data; Automatically make decisions on vehicle emergency intervention measures based on the vehicle warning feedback degree data to obtain an emergency intervention decision execution instruction set, and transmit it to the vehicle control platform to execute the vehicle driving control task.

[0012] Through the fusion of multi-modal vehicle sensing data, the present invention can perceive the changes in the vehicle's surrounding environment in real time, including factors such as roads, weather, and traffic flow, thereby enhancing the vehicle's adaptability in dynamic driving scenarios. Based on the comprehensive analysis of vehicle environment perception data, it can accurately identify and predict complex driving scenarios that the vehicle may encounter, such as high-speed driving, sudden braking, or emergency avoidance, ensuring that the monitoring system is more responsive in real time. The continuous collection and analysis of driver behavior further improve the accurate assessment of the driver's state, enabling the identification of the driver's behavior patterns, attention levels, and operating habits, and forming a driver behavior prediction model. This can not only enhance the adaptability to individual differences among drivers but also react promptly to potential dangerous behaviors such as driver fatigue and distraction. Through vehicle driving scenario simulation and risk assessment, it can simulate possible driving scenarios based on real-time environment and driver behavior prediction results, and evaluate their risks, thereby providing multi-level warning strategies for the vehicle. This dynamic adjustment based on real-time data greatly enhances the monitoring system's ability to respond to emergencies. On this basis, real-time intervention decisions can be made through a multi-level warning instruction set, reducing accidents caused by environmental changes, driver negligence, or traffic accidents. For the real-time collection and analysis of real-time warning driver behavior data and environmental perception data, it can further accurately predict the driver's behavior changes and possible risk situations, and quickly take appropriate intervention measures. The simulation and feedback evaluation of real-time driving warning scenarios can dynamically adjust the system's reaction sensitivity, ensuring that the warning system always operates with the optimal strategy and avoiding false alarms or missed alarms. Based on the analysis of the vehicle warning feedback degree, more accurate intervention decisions can be made under different driving environments and driver behaviors. By optimizing the intervention strategy, automated decision-making will greatly improve driving safety and response efficiency, achieving more intelligent and efficient vehicle control.

[0013] Optionally, step S1 is specifically as follows:

[0014] Step S11: Obtain multi-modal vehicle sensing data through a preset vehicle sensor group, and perform data preprocessing on the multi-modal vehicle sensing data to obtain multi-modal vehicle sensing data to be analyzed;

[0015] Step S12: Perform spatial classification of the multi-modal vehicle sensing data to be analyzed to obtain multi-modal sensing data inside the vehicle cabin and multi-modal sensing data outside the vehicle cabin;

[0016] Step S13: Perform sensing data fusion on the multi-modal sensing data inside the vehicle cabin and the multi-modal sensing data outside the vehicle cabin respectively to obtain fused sensing data inside the vehicle cabin and fused sensing data outside the vehicle cabin;

[0017] Step S14: Build a vehicle sensing network space based on the in-vehicle integrated sensing data and the out-of-vehicle integrated sensing data, so as to obtain an in-vehicle sensing network;

[0018] Step S15: Perform vehicle environment perception based on the in-vehicle sensing network, so as to obtain dynamic vehicle environment perception data.

[0019] Through the preset in-vehicle sensor group, the present invention obtains multi-modal in-vehicle sensing data and performs data preprocessing, which can effectively clean and integrate various sensor information from inside and outside the vehicle compartment, such as video, audio, temperature, humidity, acceleration, etc., ensuring that the acquired data is of high quality and suitable for further analysis. This process improves the accuracy and integrity of data collection, providing a reliable basis for subsequent analysis and decision-making. Classifying the data to be analyzed in space helps to more clearly distinguish the sensing information under different environmental conditions inside and outside the vehicle compartment, which not only improves the organization of the data but also facilitates subsequent optimized analysis for specific scenarios. The multi-modal sensing data inside and outside the vehicle compartment are respectively fused, which can provide a more comprehensive understanding of the vehicle interior and exterior environments and driver behavior. The generated fused data can enhance the comprehensive perception of the driver's state, the vehicle interior and exterior conditions, and the traffic environment, helping to improve the response ability and accuracy of the system. The space construction of the vehicle sensing network depends on the fused data inside and outside the vehicle compartment, which can effectively integrate and optimize different sensor information, providing stronger support for subsequent environment perception. Through this network structure, the in-vehicle system can capture and understand different driving scenarios in real time, enhancing the dynamic monitoring ability of the driver's behavior, changes in the surrounding environment, and potential risks. In addition, based on the dynamic vehicle environment perception of the in-vehicle sensing network, the system can real-time sense various influencing factors such as traffic flow, weather, and road conditions around, ensuring a comprehensive understanding of complex and changing driving scenarios. This process further improves driving safety and comfort, enabling the vehicle to more intelligently adapt to the changing road environment, make timely and accurate responses, and thus greatly improving the response efficiency and accuracy of the vehicle intelligent monitoring system.

[0020] Optionally, step S13 is specifically:

[0021] Extract the time-frequency characteristics of the multi-modal sensing data inside the vehicle compartment, so as to obtain the multi-modal sensing time-frequency characteristic data inside the vehicle compartment;

[0022] Identify the environmental noise inside the vehicle compartment according to the multi-modal sensing time-frequency characteristic data inside the vehicle compartment, so as to obtain the environmental noise data inside the vehicle compartment;

[0023] Perform environmental noise filtering on the multi-modal sensing data inside the vehicle compartment according to the environmental noise data inside the vehicle compartment, so as to obtain the multi-modal sensing data inside the vehicle compartment to be fused;

[0024] Perform principal component feature correlation on the multi-modal sensing data inside the integrated cabin to obtain the integrated sensing data inside the cabin;

[0025] Perform multi-sensor data alignment on the multi-modal sensing data outside the cabin to obtain the aligned multi-modal sensing data outside the cabin;

[0026] Conduct statistical analysis on the spatial distribution of sensors outside the cabin based on the aligned multi-modal sensing data outside the cabin to obtain the spatial distribution data of sensors outside the cabin, and construct a unified vehicle spatial coordinate system based on the spatial distribution data of sensors outside the cabin;

[0027] Perform unified scaled spatial fusion interpolation on the aligned multi-modal sensing data outside the cabin through the unified vehicle spatial coordinate system to obtain the integrated sensing data outside the cabin.

[0028] The present invention extracts time-frequency features from multi-modal sensing data in the vehicle cabin, which can reveal the time and frequency variation laws of different types of signals (such as audio, vibration, temperature, etc.) in the vehicle cabin, providing richer feature information for subsequent data analysis and pattern recognition. The time-frequency feature data not only helps to capture instantaneous changes but also can effectively distinguish different noise sources and environmental changes, thereby enhancing the sensitivity and accuracy of the system to the vehicle interior environment. Further, identifying the environmental noise in the vehicle cabin helps to accurately identify various noise sources in the vehicle, such as engine noise, in-vehicle conversation sounds, etc. This identification can provide a basis for subsequent noise filtering, thereby removing interference signals and reducing the impact of noise on data analysis and driver behavior recognition. After environmental noise filtering, the error of the multi-modal sensing data in the vehicle cabin to be fused under the influence of noise is greatly reduced, and the accuracy and reliability of the data are improved, ensuring the signal clarity in subsequent analysis. Principal component feature correlation can extract the most representative information in the vehicle cabin data, reduce redundant data, and thus further integrate multi-modal data into more efficient and information-rich fused sensing data in the vehicle cabin, providing a more valuable data source for subsequent pattern recognition and scenario assessment. Aligning the multi-modal sensing data outside the vehicle cabin enables the sensing data inside and outside the vehicle cabin to be compared and fused at the same moment, avoiding errors caused by time differences and making the perception ability of the entire vehicle more synchronous and accurate. Through the statistical analysis of the spatial distribution of sensors outside the vehicle cabin, a more comprehensive understanding of the layout and working status of external sensors can be obtained, thereby improving the overall perception ability of the external environment. Constructing a unified vehicle space coordinate system based on the spatial distribution data of sensors outside the vehicle cabin can provide a unified standard for the data fusion of all vehicle-mounted sensors, ensuring that data from different sensors can be correlated and aligned in the same coordinate system. Through unified scaled spatial fusion interpolation processing, the sensing data outside the vehicle cabin is optimized, making the fused sensing data outside the vehicle cabin more in line with the actual spatial distribution, greatly enhancing the perception ability of the vehicle-mounted system to external environmental changes, and thus providing more accurate and efficient support for the vehicle's decision-making in complex environments.

[0029] Optionally, step S2 is specifically as follows:

[0030] Step S21: Collect driver behavior data through the vehicle-mounted sensing network to obtain driver behavior data, and perform data preprocessing on the driver behavior data to obtain the driver behavior data to be analyzed;

[0031] Step S22: Extract driver behavior features based on the driver behavior data to be analyzed to obtain driver operation behavior feature data and driver behavior posture feature data;

[0032] Step S23: Identify the driving behavior pattern based on the driver's operation behavior feature data and the driver's behavior posture feature data, so as to obtain the driver behavior pattern data;

[0033] Step S24: Evaluate the focus degree of the driver behavior pattern for the driver behavior pattern data, so as to obtain the driver behavior pattern focus degree data;

[0034] Step S25: Perform temporal correlation on the dynamic vehicle environment perception data and the driver behavior pattern data, so as to obtain the behavior pattern-vehicle environment correlation data;

[0035] Step S26: Construct a driver behavior prediction model based on the behavior pattern-vehicle environment correlation data, the driver behavior pattern data, and the driver behavior pattern focus degree data.

[0036] The present invention collects driver behavior data through an in-vehicle sensing network, which can comprehensively capture the behavior data of the driver in different driving scenarios, covering information such as the driver's operating habits and behavior patterns, providing a basis for subsequent data analysis and pattern recognition. Data preprocessing can clean and standardize the collected data, remove noise and redundant information, ensure the quality of the data, and thus provide high-quality input for subsequent analysis. Based on the driver behavior data to be analyzed, behavior feature extraction can effectively extract the driver's operation behaviors and posture features respectively, providing a key basis for understanding the driver's driving style and emotional state. This feature extraction not only helps to analyze the driver's driving habits, but also can help to judge whether the driver's driving posture and behaviors meet the safe driving standards. By performing pattern recognition on the driver's operation behavior features and posture features, different driver behavior patterns in different environments can be recognized, and then a more refined driver behavior data model can be established to ensure that the diversity and individuality of driver behaviors are fully considered. The focus evaluation of the driver behavior pattern further helps to analyze the driver's attention concentration, identify the risks of fatigue driving or distracted driving that the driver may have, and improve driving safety. This evaluation can effectively capture the potential safety hazards of the driver by analyzing the driver's behavior pattern, providing effective decision-making support for subsequent safety interventions. By performing time-series correlation on the dynamic vehicle environment perception data and driver behavior pattern data, the relationship between the driver's behavior and the external environment can be revealed, and thus a more comprehensive understanding of the driving scenario can be provided. This time-series correlation helps to analyze the driver's performance under different traffic conditions, road conditions and weather conditions, so as to accurately judge whether the driver's behavior is affected by environmental factors. By constructing a driver behavior prediction model, the future behavior trend of the driver can be predicted based on multiple factors such as the driver's historical behavior pattern, environment perception data and focus, providing a basis for the decision-making of the active safety system. These analyses and predictions will help to improve the accuracy and real-time performance of the intelligent driving system, enhance the driver's safety protection ability, and provide more refined warning and intervention measures during the actual driving process.

[0037] Optionally, step S24 is specifically as follows:

[0038] Step S241: Extract the frequency-domain features of the driver behavior pattern data to obtain the driver behavior pattern frequency-domain feature data;

[0039] Step S242: Evaluate the continuity of the operation behaviors of the behavior pattern and the continuity of the driving postures of the behavior pattern for the driver behavior pattern frequency-domain feature data to obtain the operation continuity data of the behavior pattern and the posture continuity data of the behavior pattern;

[0040] Step S243: Conduct an assessment of the driving operation attention of the behavior pattern based on the continuous data of the behavior pattern operation and the continuous data of the behavior pattern posture, so as to obtain the driving operation attention data of the behavior pattern;

[0041] Step S244: Collect the in-vehicle audio data through the in-vehicle sensing network, so as to obtain the in-vehicle audio data, and conduct voice activity detection based on the in-vehicle audio data, so as to obtain the driver's voice activity data in the vehicle cabin;

[0042] Step S245: Conduct statistics on the voice activity frequency and voice emotion perception based on the driver's voice activity data in the vehicle cabin, so as to obtain the driver's voice activity frequency data and the driver's voice emotion perception data;

[0043] Step S246: Conduct an assessment of the driver's language attention based on the driver's voice activity frequency data and the driver's voice emotion perception data, so as to obtain the driver's language attention data;

[0044] Step S247: Conduct a weighted assessment of the driving concentration of the behavior pattern based on the driver's language attention data and the driving operation attention data of the behavior pattern, so as to obtain the driver's behavior pattern concentration data.

[0045] The present invention extracts frequency-domain features from driver behavior pattern data, which can effectively capture the frequency-varying features of driver behavior, thereby revealing the frequency-domain features of driver behavior in different driving scenarios. These features provide basic data for subsequent analysis of the driver's behavior pattern and driving state. On this basis, by evaluating the operation continuity and driving posture continuity of the driver behavior pattern, the behavioral coherence of the driver under different operations can be accurately understood, which helps to identify the potential fatigue state or inconsistent operations of the driver during long-term driving, and further improves the accuracy of safety monitoring. By analyzing the operation continuity data and posture continuity data of the behavior pattern to evaluate the driving operation attention, it helps to quantify the degree of concentration of the driver in different driving scenarios, so as to issue an alarm in a timely manner when the driver's attention is not concentrated. The audio data in the vehicle cabin is collected and voice activity detection is performed, which can not only capture the driver's voice activity, but also further analyze the driver's emotional state and language response through voice activity frequency statistics and voice emotion perception. This provides effective data support for evaluating the driver's mental state and emotional fluctuations, and helps to identify whether the driver's driving behavior is affected by emotional problems. Through the comprehensive analysis of the driver's voice activity frequency data and voice emotion perception data, the driver's language attention can be effectively evaluated, and the attention concentration of the driver in communication and interaction can be revealed. Based on the weighted evaluation of the driver's language attention data and the driving operation attention data of the behavior pattern, the behavioral concentration of the driver and the attention level in voice interaction can be comprehensively considered, providing a more comprehensive and accurate evaluation of the driver's concentration. This comprehensive evaluation can not only improve driving safety, but also provide a more accurate decision-making basis for the intelligent driving system, especially when emergency intervention on the driver's state is required, ensuring that the system can respond in a timely manner and perform effective intervention.

[0046] Optionally, step S3 is specifically as follows:

[0047] Step S31: Perform Monte Carlo vehicle environment simulation based on the dynamic vehicle environment perception data to obtain dynamic vehicle environment simulation data;

[0048] Step S32: Obtain the vehicle rated parameter set and construct a vehicle dynamics model based on the vehicle rated parameter set;

[0049] Step S33: Perform driver behavior prediction on the driver behavior data to be analyzed and the dynamic vehicle environment perception data through the driver behavior prediction model to obtain driver behavior prediction data;

[0050] Step S34: Perform vehicle driving scenario simulation on the driver behavior prediction data and the dynamic vehicle environment simulation data based on the vehicle dynamics model to obtain vehicle driving scenario simulation data;

[0051] Step S35: Conduct a driving scenario risk assessment on the vehicle driving scenario simulation data to obtain driving scenario risk assessment data.

[0052] Through Monte Carlo vehicle environment simulation with dynamic vehicle environment perception data, the present invention can provide the system with various possible environmental change scenarios, enhancing the system's response ability in an uncertain environment. This simulation can generate data covering various possible road conditions, weather changes, and traffic situations, helping the system better understand and predict actual driving scenarios, and thus providing a more accurate basis for risk assessment. Constructing a vehicle dynamics model based on the vehicle's rated parameter set is to accurately simulate the actual performance of the vehicle under different environments and driving behaviors, ensuring that the simulation data can reflect the motion characteristics and performance of the vehicle during actual driving, which is crucial for subsequent scenario simulation and risk assessment. By predicting driver behavior data and dynamic vehicle environment perception data through a driver behavior prediction model, it is possible to identify in advance the behavior trends of the driver and the possible changes the vehicle may face in a specific environment, providing data support for early response to potential dangers and emergencies. This prediction not only helps understand the driver's current state but also effectively provides a basis for the system to formulate more personalized safety warnings and intervention measures. On this basis, combining the vehicle dynamics model and the predicted driver behavior data for vehicle driving scenario simulation can more accurately restore the dynamic changes of the vehicle in a specific scenario, providing rich scenario data for risk assessment and helping the system identify potential safety hazards and complex driving scenarios. By conducting a driving scenario risk assessment on the vehicle driving scenario simulation data, it is possible to quantify the risk levels in different driving scenarios, accurately evaluate the impact of driver behavior and environmental factors on driving safety, enabling the system to dynamically adapt to different driving environments, thereby identifying and avoiding potential dangers in advance and ensuring driving safety.

[0053] Optionally, step S35 is specifically as follows:

[0054] Step S351: Extract features from the vehicle driving scenario simulation data to obtain vehicle trajectory simulation data and driver behavior simulation data;

[0055] Step S352: Conduct a statistical analysis of the vehicle collision probability distribution based on the vehicle trajectory simulation data to obtain vehicle collision probability distribution data;

[0056] Step S353: Evaluate the driver behavior focus of the driver behavior simulation data based on the driver behavior pattern focus data to obtain driver simulation behavior focus data;

[0057] Step S354: Calculate the vehicle trajectory correction probability based on the vehicle dynamics model for the vehicle collision probability distribution data to obtain vehicle trajectory correction probability data;

[0058] Step S355: Conduct a weighted assessment of the driver's simulated behavior concentration data and the vehicle trajectory correction probability data for the driving scenario vehicle collision risk, so as to obtain the driving scenario risk assessment data.

[0059] Through feature extraction of the vehicle driving scenario simulation data, the present invention can obtain detailed information on the vehicle trajectory and driver behavior from multiple dimensions, which provides rich basic data for further risk assessment. The vehicle trajectory simulation data can accurately depict the driving path of the vehicle in a specific scenario, while the driver behavior simulation data helps to capture the driver's operation behavior and reaction mode, so as to more comprehensively understand the driving scenario. Based on the vehicle trajectory simulation data, statistical analysis of the vehicle collision probability distribution can be carried out to statistically analyze the risks of vehicle collisions in different driving environments and scenarios, and generate a distribution data of the collision probability, providing a quantitative basis for subsequent safety analysis. The evaluation of the driver behavior pattern concentration data can accurately reflect the driver's concentration state during driving. Through the concentration analysis of the driver behavior simulation data, it helps to identify the possible decrease in attention or fatigue behavior of the driver in a specific scenario, which is crucial for preventing safety incidents caused by driver errors. Calculating the trajectory correction probability based on the vehicle dynamics model for the vehicle collision probability distribution data helps to dynamically adjust the driving trajectory of the vehicle during the simulation process and reduce the possible collision risk. By calculating the possible trajectory correction probability of the vehicle in different scenarios, an optimization plan for the safe driving path of the vehicle can be provided for the system, helping the vehicle to better cope with complex road environments. Conducting a weighted assessment of the driver's simulated behavior concentration data and the vehicle trajectory correction probability data helps to comprehensively evaluate the collision risk in the driving scenario, ensuring that potential hazards can be identified and response decisions can be made in a timely manner in different situations. This comprehensive assessment enables the system to accurately evaluate and predict the comprehensive risks of the driver and the vehicle during real-time driving, further enhancing driving safety.

[0060] Optionally, step S4 is specifically as follows:

[0061] Step S41: Perform real-time vehicle environment perception and real-time driver behavior collection through the in-vehicle sensing network, so as to obtain real-time vehicle environment perception data and real-time driver behavior data;

[0062] Step S42: Integrate the real-time vehicle environment perception data and the real-time driver behavior data based on the vehicle dynamics model to obtain real-time vehicle driving scenario data;

[0063] Step S43: Calculate the scenario similarity according to the vehicle driving scenario simulation data and the real-time vehicle driving scenario data, so as to obtain the driving scenario similarity data;

[0064] Step S44: Extract vehicle warning trigger parameters from the vehicle rated parameter group to obtain a vehicle warning trigger parameter group, and allocate risk-level vehicle warning instructions to the vehicle warning trigger parameter group according to the driving scenario risk assessment data, so as to obtain a multi-risk-level warning trigger parameter group;

[0065] Step S45: Utilize the driving scenario similarity data, and perform real-time driving scenario risk assessment on the real-time vehicle driving scenario data based on the driving scenario risk assessment data, so as to obtain real-time driving scenario risk assessment data;

[0066] Step S46: Associate warning trigger instructions with the multi-risk-level warning trigger parameter group through a predefined warning instruction set, so as to obtain a multi-level warning trigger instruction set;

[0067] Step S47: Match warning instructions to the real-time driving scenario risk assessment data based on the multi-level warning trigger instruction set, so as to obtain a vehicle warning instruction set, and transmit it to the vehicle control platform to execute the vehicle driving warning task.

[0068] The present invention conducts real-time vehicle environment perception and driver behavior collection through an in-vehicle sensing network, capable of continuously monitoring and obtaining real-time state data of the vehicle in a dynamic environment, and timely acquiring the driver's behavior data. This real-time monitoring can provide the most accurate and up-to-date information for subsequent driving scenario analysis, ensuring timely identification and response to potential risks in complex driving environments. By integrating real-time data based on a vehicle dynamics model, the driver's behavior can be effectively combined with the current road environment, thereby generating a comprehensive vehicle driving scenario data to provide comprehensive scenario support for risk assessment and decision-making. By calculating the scenario similarity between the vehicle driving scenario simulation data and real-time data, the similarity between historical data and the current scenario can be compared and analyzed to accurately predict possible driving risks, providing a scientific basis for subsequent early warning decisions. Extracting early warning trigger parameters from the vehicle's rated parameter group and combining them with the driving scenario risk assessment data for risk level instruction allocation can achieve dynamic setting of early warning trigger parameters based on the actual driving environment, enabling the system to adapt to the requirements in different driving scenarios, thereby effectively improving the accuracy and response speed of early warnings. Using the driving scenario similarity data to conduct risk assessment on the real-time driving scenario can continuously update the driving scenario risk assessment data, enabling the system to continuously grasp the potential risks of the vehicle, quickly identify dangerous situations, and quantitatively evaluate the severity of the danger. Based on the association between the early warning trigger parameter group and early warning instruction set of multiple risk levels, appropriate early warning instructions can be accurately matched for different levels of risk situations, and real-time and accurate decisions can be made in multiple dangerous situations. After the early warning instructions are matched and transmitted to the vehicle control platform, the early warning task and emergency intervention measures can be promptly initiated to ensure that the driver can receive timely safety prompts or the vehicle can automatically intervene, greatly improving driving safety and the vehicle's ability to respond to sudden risks.

[0069] Optionally, step S6 is specifically as follows:

[0070] Step S61: Calculate the driving scenario similarity based on the vehicle driving scenario simulation data and the real-time driving early warning scenario simulation data, so as to obtain the real-time driving early warning scenario similarity data;

[0071] Step S62: Use the real-time driving early warning scenario similarity data and conduct real-time driving early warning scenario risk assessment on the real-time driving early warning scenario simulation data based on the driving scenario risk assessment data, so as to obtain the real-time driving early warning scenario risk assessment data;

[0072] Step S63: Conduct a risk assessment comparison on the real-time driving scenario risk assessment data and the real-time driving early warning scenario risk assessment data, so as to obtain the risk assessment error data;

[0073] Step S64: Perform a driver behavior focus evaluation on the real-time warning driver behavior data and the real-time warning vehicle environment perception data according to the driver behavior pattern focus data, so as to obtain the real-time driver behavior focus data;

[0074] Step S65: Perform a weighted evaluation of the vehicle warning feedback degree based on the risk assessment error data and the real-time driver behavior focus data, so as to obtain the vehicle warning feedback degree data;

[0075] Step S66: Select an emergency intervention instruction from the multi-level warning trigger instruction set according to the vehicle warning feedback degree data, so as to obtain an emergency intervention decision execution instruction set, and transmit it to the vehicle control platform to execute the vehicle driving control task.

[0076] The present invention calculates the driving scenario similarity by combining the vehicle driving scenario simulation data and the real-time warning scenario simulation data, effectively identifies the similarity between the current driving scenario and the historical warning scenario, and provides a scientific basis for potential dangers. Utilize the similarity data for real-time risk assessment to ensure that the system dynamically assesses driving risks based on the latest data and promptly discovers potential safety hazards. By comparing the real-time driving scenario risk assessment data with the warning scenario risk assessment data, optimize the assessment accuracy, reduce misjudgments and lower the probability of false warnings. Combine the driver behavior pattern focus data to evaluate the driver's attention, identify fatigue or distracted driving, and optimize the warning timing and method. Based on the risk assessment error data and the driver focus data, perform a weighted evaluation of the warning feedback degree to improve the warning accuracy and reduce unnecessary warnings. Automatically select emergency intervention instructions according to the warning feedback degree data to ensure that appropriate vehicle driving control tasks are executed in case of emergencies, reduce accidents, and ensure safety.

[0077] Optionally, this specification also provides a data model processing system for executing the data model processing method described above. The data model processing system includes:

[0078] A vehicle environment perception module, configured to obtain multi-modal vehicle-mounted sensing data, perform multi-modal sensing data fusion according to the multi-modal vehicle-mounted sensing data to obtain a vehicle-mounted sensing network; perform vehicle environment perception according to the vehicle-mounted sensing network to obtain dynamic vehicle environment perception data;

[0079] A driver behavior pattern modeling module, configured to collect driver behavior data through the vehicle-mounted sensing network to obtain driver behavior data, perform driver behavior pattern recognition according to the driver behavior data to obtain driver behavior pattern data; perform behavior pattern modeling on the driver behavior pattern data to obtain a driver behavior prediction model;

[0080] A driving scenario risk assessment module, which is used to simulate a vehicle driving scenario based on a driver behavior prediction model and dynamic vehicle environment perception data, so as to obtain vehicle driving scenario simulation data, and perform a driving scenario risk assessment on the vehicle driving scenario simulation data, so as to obtain driving scenario risk assessment data;

[0081] An early warning strategy analysis module, which is used to analyze a multi-level vehicle early warning strategy according to the driving scenario risk assessment data, so as to obtain a vehicle early warning instruction set, and transmit it to the vehicle control platform to execute the vehicle driving early warning task;

[0082] A driver behavior prediction module, which is used to collect real-time driver behavior data and real-time vehicle environment perception through an in-vehicle sensor network, so as to obtain real-time early warning driver behavior data and real-time early warning vehicle environment perception data, and simulate a vehicle driving early warning scenario for the real-time early warning driver behavior data and the real-time early warning vehicle environment perception data through a driver behavior prediction model, so as to obtain real-time driving early warning scenario simulation data;

[0083] An early warning feedback degree evaluation module, which is used to evaluate the vehicle early warning feedback degree for the real-time driving early warning scenario simulation data, so as to obtain vehicle early warning feedback degree data; automatically make a decision on vehicle emergency intervention measures according to the vehicle early warning feedback degree data, so as to obtain an emergency intervention decision execution instruction set, and transmit it to the vehicle control platform to execute the vehicle driving control task.

[0084] The data model processing system of the present invention, which can implement any data model processing method of the present invention, and is used as a medium for coordinating the operations and signal transmissions between each module to complete the data model processing method. The internal modules of the system cooperate with each other, thereby improving driving safety and response efficiency. Brief Description of the Drawings

[0085] By reading the detailed description of the non-restrictive embodiments with reference to the following drawings, other features, objectives and advantages of the present invention will become more obvious:

[0086] Figure 1 It is a schematic flowchart of the steps of the data model processing method of the present invention;

[0087] Figure 2 It is a detailed schematic flowchart of step S1 in the present invention;

[0088] The implementation, functional characteristics and advantages of the objectives of the present invention will be further described with reference to the embodiments and the drawings. Detailed Embodiments

[0089] The technical method of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0090] In addition, the accompanying drawings are only schematic diagrams of the present invention and are not necessarily drawn to scale. The same reference numerals in the drawings represent the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor methods and / or microcontroller methods.

[0091] It should be understood that although terms such as "first", "second", etc. may be used here to describe each unit, these units should not be limited by these terms. These terms are only used to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, the first unit can be called the second unit, and similarly the second unit can be called the first unit. The term "and / or" used here includes any and all combinations of one or more of the listed associated items.

[0092] To achieve the above object, please refer to Figures 1 to 2 , the present invention provides a data model processing method, and the method includes the following steps:

[0093] Step S1: Obtain multi-modal vehicle sensing data, and perform multi-modal sensing data fusion according to the multi-modal vehicle sensing data to obtain a vehicle sensing network; perform vehicle environment perception according to the vehicle sensing network to obtain dynamic vehicle environment perception data;

[0094] In this embodiment, real-time data of multiple sensor data sources are obtained through an in-vehicle sensor group, including cameras, radars, lidars (LiDARs), infrared sensors, etc. These data involve perception information of the vehicle's surrounding environment, such as road conditions, traffic flow, obstacles, etc. Then, these data from different sensors are fused through a multi-modal data fusion algorithm (such as the weighted average method or the Kalman filtering algorithm) to obtain unified in-vehicle sensing network data. This data set reflects dynamic vehicle environment perception data, including speed, direction, distance, etc. Further, by analyzing these data, a comprehensive perception of the vehicle's surrounding environment is achieved. Specifically, if a camera detects an obstacle ahead and the lidar provides accurate distance information, the fused data can provide a more accurate real-time state of the vehicle's position and the surrounding environment.

[0095] Step S2: Collect driver behavior data through the in-vehicle sensing network to obtain driver behavior data, and perform driver behavior pattern recognition based on the driver behavior data to obtain driver behavior pattern data; perform behavior pattern modeling on the driver behavior pattern data to obtain a driver behavior prediction model;

[0096] In this embodiment, through the in-vehicle sensor network, driver behavior data such as steering angle, accelerator pedal position, brake state, and vehicle speed are collected. After these data are preprocessed (such as noise filtering, data standardization, etc.), through machine learning algorithms (such as support vector machines, decision trees, etc.), the driver's behavior patterns can be recognized, such as normal driving, emergency braking, excessive acceleration, etc. Based on these behavior pattern data, a statistical modeling method (such as the hidden Markov model) is used for behavior pattern modeling to obtain a driver behavior prediction model. Through this model, the system can predict the driver's future operation behavior, thereby providing a basis for subsequent driving scenario simulation and risk assessment.

[0097] Step S3: Perform vehicle driving scenario simulation based on the driver behavior prediction model and the dynamic vehicle environment perception data to obtain vehicle driving scenario simulation data, and perform driving scenario risk assessment on the vehicle driving scenario simulation data to obtain driving scenario risk assessment data;

[0098] In this embodiment, the trained driver behavior prediction model and dynamic vehicle environment perception data are used in combination with the vehicle dynamics model (such as a four-wheel model, a vehicle driving state model, etc.) to simulate the driving scenario. By simulating different driving scenarios (such as normal driving, emergency braking, emergency avoidance, etc.), corresponding vehicle driving scenario simulation data can be generated. These simulation data reflect the vehicle motion state under different environments and driver behaviors. Risk assessment is performed on these simulation data, and the collision risk, emergency stop risk, etc. are quantitatively assessed using methods such as Monte Carlo simulation, so as to obtain driving scenario risk assessment data. For example, in a complex urban traffic environment, the potential risk of the interaction between the driver's behavior and the surrounding environment can be assessed, thereby providing decision support for the vehicle's early warning strategy.

[0099] Step S4: Performing a vehicle multi-level warning strategy analysis based on the driving scenario risk assessment data, thereby obtaining a vehicle warning instruction set, and transmitting it to the vehicle control platform to execute the vehicle driving warning task;

[0100] In this embodiment, based on the obtained driving scenario risk assessment data, a multi-level early warning strategy analysis is performed on the risks of different driving scenarios through algorithm analysis (such as multi-level decision trees or fuzzy logic reasoning). These early warning strategies divide the risk level into low, medium and high levels according to the size of the risk, the driver's behavior pattern and environmental factors. For example, when the assessment results show that the collision risk is high, the risk level will be set to high and a corresponding early warning instruction set will be generated. These instructions include but are not limited to sound warnings, brake prompts, steering assistance, etc. The instruction set is then transmitted to the vehicle control platform, and the vehicle control system performs the corresponding driving warning tasks according to the early warning instructions.

[0101] Step S5: collecting real-time driver behavior data and real-time vehicle environment perception through the vehicle-mounted sensor network, thereby obtaining real-time warning driver behavior data and real-time warning vehicle environment perception data, and performing driver behavior prediction on the real-time warning driver behavior data and real-time warning vehicle environment perception data through a driver behavior prediction model, thereby obtaining real-time warning driver behavior prediction data;

[0102] In this embodiment, the on-board sensor network continues to collect real-time driver behavior data and vehicle environment perception data, such as the driver's steering behavior, acceleration or deceleration operations, the location of surrounding vehicles, etc. Combined with the driver behavior prediction model, the driver's possible next behavior is predicted, such as whether to suddenly accelerate or brake. These prediction data help to update the driving scenario status in real time and provide real-time basis for subsequent driver behavior predictions. Through real-time collected data and prediction models, it is possible to respond to the driver's intentions and make predictions in an instant, thereby improving the response speed and accuracy of the early warning system.

[0103] Step S6: Evaluate the vehicle warning feedback degree for the real-time driving warning scenario simulation data to obtain vehicle warning feedback degree data; automatically make a decision on the vehicle emergency intervention measures based on the vehicle warning feedback degree data to obtain an emergency intervention decision execution instruction set, and transmit it to the vehicle control platform to execute the vehicle driving control task.

[0104] In this embodiment, the real-time driving warning scenario simulation data is used to evaluate the warning feedback degree of the vehicle. This process is based on the previously calculated risk assessment data, combined with the real-time state of the vehicle and the execution effect of the warning instruction, to evaluate the feedback degree of the vehicle's current warning system. If the feedback effect of the warning system is good, the warning feedback degree data is high; otherwise, it is low. According to the feedback degree data, automatically decide whether to perform emergency intervention, such as operations like automatic braking, acceleration, or steering wheel adjustment. If the feedback degree is low, an emergency intervention decision will be triggered and an execution instruction set will be generated, such as sending a braking instruction or a steering instruction to the vehicle control platform, so as to ensure the safety of the driver.

[0105] Optionally, step S1 is specifically as follows:

[0106] Step S11: Obtain multi-modal vehicle-borne sensing data through a preset vehicle-borne sensor group, and perform data preprocessing on the multi-modal vehicle-borne sensing data to obtain multi-modal vehicle-borne sensing data to be analyzed;

[0107] In this embodiment, a variety of vehicle-borne sensing data is obtained through a preset vehicle-borne sensor group. Specifically, the sensor group can include an environmental camera, radar, lidar, temperature and humidity sensor, ultrasonic sensor, etc., which can collect the environmental data inside and outside the vehicle cabin in real time. These data include vehicle speed, the distance to obstacles around the vehicle, the temperature and humidity inside the vehicle, and the physiological state of the driver, etc. To reduce the influence of noise and improve the effectiveness of the data, all sensing data is preprocessed, including steps such as removing outliers, signal smoothing, and normalization processing, to ensure the accuracy and stability of the data, and finally generate multi-modal vehicle-borne sensing data to be analyzed.

[0108] Step S12: Perform spatial classification on the multi-modal vehicle-borne sensing data to be analyzed to obtain multi-modal sensing data inside the vehicle cabin and multi-modal sensing data outside the vehicle cabin;

[0109] In this embodiment, spatial classification is performed on the multi-modal vehicle sensing data to be analyzed, aiming to distinguish the data inside and outside the vehicle cabin. The data inside the vehicle cabin mainly refers to driver behavior, in-vehicle climate data, in-vehicle images, etc., while the data outside the vehicle cabin mainly involves the environmental perception around the vehicle, such as road conditions, traffic flow, obstacle detection, etc. Through a spatial classification algorithm (such as a classification method based on sensor position), the data inside and outside the vehicle cabin can be effectively distinguished, and clearer structured information can be provided for subsequent data fusion processing. For example, by calculating the relative positions of the camera and the radar, the data inside and outside the vehicle can be automatically identified and classified, laying a foundation for subsequent data fusion.

[0110] Step S13: Perform sensor data fusion on the multi-modal sensing data inside the vehicle cabin and the multi-modal sensing data outside the vehicle cabin respectively, so as to obtain the fused sensing data inside the vehicle cabin and the fused sensing data outside the vehicle cabin;

[0111] In this embodiment, the multi-modal sensing data inside the vehicle cabin and the multi-modal sensing data outside the vehicle cabin are fused separately. The sensing data inside the vehicle cabin may include the steering angle of the driver, vehicle speed, temperature and humidity sensor data, etc., while the data outside the vehicle cabin may include the obstacle position provided by lidar, traffic signs detected by the camera, etc. The data fusion method adopts the weighted average method, Kalman filter algorithm or deep learning model, etc., and weighted fusion is performed according to the accuracy of the sensor and the data type to obtain more accurate fused data. For example, based on the distance of the obstacle detected by the radar, the shape and type of the obstacle are further confirmed by the camera, and finally more accurate environmental perception data is synthesized.

[0112] Step S14: Build a vehicle sensing network space according to the fused sensing data inside the vehicle cabin and the fused sensing data outside the vehicle cabin, so as to obtain a vehicle-mounted sensing network;

[0113] In this embodiment, the functions of the in-vehicle integrated sensing data and the out-of-vehicle integrated sensing data are combined with each other, which can provide comprehensive environmental perception and driver behavior judgment, and lay a foundation for the intelligent decision-making of the vehicle. In specific implementation, the in-vehicle integrated sensing data integrates the data such as driver behavior, vehicle speed, and driver posture collected by in-vehicle sensors (such as seat sensors, in-vehicle cameras, vehicle speed sensors, etc.), and can accurately reflect the state and intention of the driver. For example, by analyzing the driver's steering angle, acceleration, and braking behaviors, etc., the system can predict whether the driver is in a tense or inattentive state, and thus issue a warning or perform corresponding intervention in advance. The out-of-vehicle integrated sensing data, on the other hand, provides comprehensive environmental perception support by integrating the surrounding environmental information obtained by out-of-vehicle sensors (such as lidar, front cameras, ultrasonic sensors, etc.), such as the distance to obstacles, road conditions, traffic signs, etc. These data can help the vehicle to identify obstacles ahead, the relative speed of other vehicles, traffic signals, etc. in real time, and provide an accurate basis for the intelligent path planning and collision warning system. In the process of constructing the vehicle-mounted sensing network, the in-vehicle and out-of-vehicle integrated data establish a vehicle-mounted network framework through the data communication protocol of the vehicle-mounted computing platform (such as CAN bus, Ethernet, etc.), and form a unified environmental perception network by combining the real-time dynamic data of the driver behavior and the surrounding environment. Through data fusion, the in-vehicle data helps the system to understand the driver's intention and state, and the out-of-vehicle data provides the surrounding environmental information. After the two are combined, it can provide more accurate driving decision support for the system. For example, in the event of a sudden traffic situation, the system can analyze the driver's reaction according to the in-vehicle integrated data, and then combine the environmental information provided by the out-of-vehicle integrated data to automatically make the most appropriate emergency response or path adjustment decision.

[0114] Step S15: Perform vehicle environmental perception according to the vehicle-mounted sensing network, so as to obtain dynamic vehicle environmental perception data.

[0115] In this embodiment, vehicle environmental perception is performed based on the established vehicle-mounted sensing network. This process analyzes the data obtained by the vehicle-mounted sensing network to perceive the dynamic environment around the vehicle in real time. Using the data of the vehicle-mounted sensing network, including information such as vehicle speed, the distance to surrounding obstacles, and traffic signals, the vehicle environmental perception system can timely identify changes in the environment, such as traffic congestion ahead and road obstacles. At the same time, the driver's behavior pattern can be predicted based on these data to determine whether warning or intervention is required. For example, if an obstacle is detected ahead and the driver does not take evasive actions, the environmental perception system can issue a warning signal to remind the driver to pay attention to safety.

[0116] Optionally, step S13 is specifically:

[0117] Extract the time-frequency features of the multi-modal sensing data in the vehicle cabin to obtain the multi-modal sensing time-frequency feature data in the vehicle cabin;

[0118] In this embodiment, the time-frequency features of the multi-modal sensing data in the vehicle cabin are extracted. Specifically, the multi-modal sensors in the vehicle cabin include accelerometers, gyroscopes, seat sensors, vehicle speed sensors, in-vehicle cameras, etc. These sensors collect vehicle state and driver behavior data in real time. By using the Short-Time Fourier Transform (STFT) technique, the signals collected by the accelerometers and gyroscopes are subjected to time-frequency analysis to extract the energy distribution and time-domain variation characteristics in specific frequency bands, such as the change trends of vehicle acceleration and vibration. These time-frequency features can reflect dynamic behaviors such as vibration, acceleration and deceleration, and steering wheel rotation during vehicle driving, thus providing effective feature data for subsequent environmental noise recognition and processing.

[0119] Identify the environmental noise in the vehicle cabin based on the multi-modal sensing time-frequency feature data in the vehicle cabin to obtain the environmental noise data in the vehicle cabin;

[0120] In this embodiment, after the extraction of the multi-modal sensing time-frequency feature data in the vehicle cabin, the environmental noise recognition algorithm is then used to identify these feature data. The specific operations include classifying the in-vehicle environmental noise (such as engine noise, external traffic noise, in-vehicle air-conditioning noise, etc.). Using machine learning methods, such as Support Vector Machine (SVM) or Convolutional Neural Network (CNN), a noise classification model is trained based on the time-frequency feature data to identify different types of in-vehicle noise. The identification process involves clustering analysis of the time-frequency feature data to accurately locate the noise source. For example, the engine sound when the vehicle starts is identified as noise. Through this process, the noise data in the vehicle cabin can be obtained.

[0121] Perform environmental noise filtering on the multi-modal sensing data in the vehicle cabin according to the environmental noise data in the vehicle cabin to obtain the multi-modal sensing data in the vehicle cabin to be fused;

[0122] In this embodiment, after the identification of the environmental noise data in the vehicle cabin, a filtering algorithm is used to filter the noise of the multi-modal sensing data in the vehicle cabin. Specifically, a suitable filtering algorithm (such as Kalman filtering, waveform denoising, etc.) is selected, and the noise frequency characteristics in the vehicle cabin noise data are used to remove the influence on the vehicle cabin data. When designing the filter, based on the identified noise frequency band (such as low-frequency noise), this part of the interference is filtered out from the in-vehicle sensor data, and the effective driver behavior and vehicle state information are retained. Finally, the processed data becomes the multi-modal sensing data in the vehicle cabin to be fused, ensuring the data quality while avoiding noise interference.

[0123] Perform principal component feature correlation on the multi-modal sensing data in the vehicle cabin to be fused to obtain the fused sensing data in the vehicle cabin;

[0124] In this embodiment, after noise filtering is completed, principal component feature correlation is performed on the multi-modal sensing data inside the vehicle cabin to be fused. The principal component analysis (PCA) method is applied to multiple data dimensions such as vehicle speed, seat sensor data, and driver posture. Through linear transformation of the data, the most representative principal component features are extracted. In implementation, through dimensionality reduction technology, the high-dimensional sensing data is transformed into a low-dimensional feature set, which not only retains the key behavior pattern information but also reduces data redundancy and improves the efficiency and accuracy of subsequent data processing.

[0125] Perform multi-sensor data alignment on the multi-modal sensing data outside the vehicle cabin to obtain aligned multi-modal sensing data outside the vehicle cabin;

[0126] In this embodiment, multi-sensor data alignment is performed on the multi-modal sensing data outside the vehicle cabin. Time synchronization technology and spatial alignment methods are used to ensure that the data collected by different sensors can be aligned both in time and space. The data collected by lidar, ultrasonic sensors, front cameras, etc. installed outside the vehicle need to undergo timestamp calibration and spatial position alignment to ensure that the data of these sensors can be comprehensively analyzed within the same time window. For example, the distance data of lidar and the distance data of ultrasonic waves need to be aligned through a time synchronization algorithm so that they can reflect the environmental information around the vehicle at the same moment.

[0127] Perform statistics on the spatial distribution of sensors outside the vehicle cabin based on the aligned multi-modal sensing data outside the vehicle cabin to obtain spatial distribution data of sensors outside the vehicle cabin, and construct a unified spatial coordinate system for the vehicle based on the spatial distribution data of sensors outside the vehicle cabin;

[0128] In this embodiment, for the aligned multi-modal sensing data outside the vehicle cabin, statistics on the spatial distribution of sensors outside the vehicle cabin are performed. This process constructs a distribution model of the external sensors of the vehicle by measuring the spatial distribution characteristics of different sensors. For example, by analyzing the spatial positions of external lidar, cameras, and ultrasonic sensors, their observation ranges and coverage areas are determined, and statistical analysis is performed on their spatial distribution data. These data help to understand the working range and blind spots of each sensor outside the vehicle and further optimize the vehicle's perception ability. Based on the spatial relationships in the spatial distribution data of sensors outside the vehicle cabin, a unified spatial coordinate system is constructed to ensure the accuracy of data fusion.

[0129] Perform unified scaled spatial fusion interpolation on the aligned multi-modal sensing data outside the vehicle cabin through the unified spatial coordinate system of the vehicle to obtain fused sensing data outside the vehicle cabin.

[0130] In this embodiment, based on the statistical analysis of the spatial distribution of sensors outside the vehicle cabin, a unified spatial coordinate system is used to perform scaled spatial fusion interpolation on multi-modal sensing data outside the vehicle cabin. Through a spatial interpolation algorithm (such as Kriging interpolation or cubic interpolation), the sensor data is uniformly scaled so that the data from different sensors can be fused under the same scale and coordinate system. The specific operations include performing spatial interpolation on the lidar, ultrasonic sensor, and camera data to ensure that the fusion results in the unified coordinate system can truly and accurately reflect the spatial relationship between the vehicle and the surrounding environment. This process helps to enhance the vehicle's perception ability in complex environments and provide more accurate collision warning and path planning information.

[0131] Optionally, step S2 is specifically as follows:

[0132] Step S21: Collect driver behavior data through the in-vehicle sensing network to obtain driver behavior data, and perform data preprocessing on the driver behavior data to obtain the driver behavior data to be analyzed;

[0133] In this embodiment, the in-vehicle sensing network includes multiple sensors, such as seat sensors, vehicle speed sensors, in-vehicle cameras, steering wheel sensors, etc. The behavior data of the driver is collected in real time through these sensors. Specifically, the vehicle speed sensor records the driving speed of the vehicle, the seat sensor captures the driver's posture and sitting position information, the in-vehicle camera monitors the driver's eye movement and facial expressions, and the steering wheel sensor records the driver's steering behavior. After data collection, the obtained driver behavior data is preprocessed to remove noise, fill in missing values, and perform standardization processing to obtain the driver behavior data to be analyzed. These data provide the basis for subsequent feature extraction and analysis.

[0134] Step S22: Extract driver behavior characteristics based on the driver behavior data to be analyzed to obtain driver operation behavior characteristic data and driver behavior posture characteristic data;

[0135] In this embodiment, after obtaining the driver behavior data to be analyzed, the operation behavior features are first extracted. The operation behavior features include steering angle change, acceleration, braking force and frequency, etc. For example, through the data collected by the steering wheel sensor, the steering behavior of the driver in different driving scenarios is analyzed, such as whether there are sharp turns when driving at high speed, or whether the driver frequently changes lanes in traffic congestion. In addition, the behavior posture features of the driver are also extracted. The sitting posture and body posture of the driver are analyzed through the seat sensor, such as whether the body is tilted or in a fatigued state, which may affect the driver's reaction. By integrating eye movement tracking and facial expression recognition data, it is also possible to identify whether the driver is concentrating on the road or showing signs of distraction or fatigue. The extraction of all these operation and posture features provides an accurate data basis for subsequent behavior pattern recognition.

[0136] Step S23: Based on the driver operation behavior feature data and the driver behavior posture feature data, perform driving behavior pattern recognition to obtain driver behavior pattern data;

[0137] In this embodiment, based on the driver operation behavior features and posture feature data, a deep learning-based driving behavior pattern recognition method is used for analysis. Specifically, through a neural network (such as a convolutional neural network or a recurrent neural network), the steering, acceleration, and braking data of the driver are trained with their eye movement and facial expressions to identify various driving behavior patterns, such as normal driving, speeding, hard braking, fatigue driving, etc. During the training process, the model also considers the driver's physiological state (such as heart rate, body temperature, etc.), classifies various driving patterns using the labeled data set, and finally obtains the driver's behavior pattern data through training. For example, if the driver frequently accelerates and hard brakes, and the eye movement trajectory shows a distracted state, the system can identify that the driver is in the "distracted driving" mode and issue a corresponding warning.

[0138] Step S24: Evaluate the driver behavior pattern focus of the driver behavior pattern data to obtain driver behavior pattern focus data;

[0139] In this embodiment, after the driver behavior pattern data is recognized, the concentration is further evaluated. This evaluation combines the driver's physiological data (such as heart rate, skin conductance response), behavioral data (such as eye movement, seat pressure distribution), and driving operation characteristics (such as braking and acceleration force). In a specific implementation, a heart rate monitor and a skin conductance sensor are used to obtain the driver's physiological response, and the driver's sitting posture change is obtained through a seat sensor. Further, by combining eye movement tracking data and facial expression analysis, it is determined whether the driver is in a state of high fatigue or distraction. For example, if the heart rate increases significantly and the driver's head shakes frequently or the eyes close frequently, it indicates that the driver may be fatigued and the concentration score will decrease. Finally, the evaluation result is converted into driver behavior pattern concentration data, which will become an important input for driver behavior prediction.

[0140] Step S25: Perform temporal association on the dynamic vehicle environment perception data and the driver behavior pattern data to obtain behavior pattern-vehicle environment association data;

[0141] In this embodiment, after obtaining the driver behavior pattern data and the concentration data, the driver behavior pattern is associated with the dynamic vehicle environment data (such as vehicle distance, road conditions, traffic signals, etc.) through a temporal association algorithm. The dynamic vehicle environment data is collected in real time by in-vehicle sensors (such as lidar, forward camera, millimeter wave radar) to obtain the distance to the obstacle in front, the speed of surrounding vehicles, and the status of traffic lights. Specifically, by establishing a temporal relationship model between the driver's behavior and the vehicle surrounding environment, the driver's acceleration / braking behavior is matched with the road conditions (such as speed limit, traffic congestion, etc.). If the system detects a sudden change in the driver's driving behavior (such as sudden braking), it is associated with the environmental data (such as an obstacle in front), and it is determined whether there is a potential danger, and on this basis, a driving behavior-environment association data set is generated.

[0142] Step S26: Construct a driver behavior prediction model based on the behavior pattern-vehicle environment association data, the driver behavior pattern data, and the driver behavior pattern concentration data.

[0143] In this embodiment, a driver behavior prediction model is constructed by combining driver behavior pattern data, concentration data, and behavior-environment association data. This model uses machine learning algorithms (such as decision trees, neural networks, etc.) to perform feature fusion on driver behavior pattern data, concentration data, and behavior-environment association data, and uses the fused features as input. Through continuous iteration and training, it can predict the driver's reactions in different situations. For example, if it is detected that the driver has entered the highway and there has been no obvious reaction for a long time, the prediction model may predict that the driver is at risk of fatigue or distraction, and then take preventive measures in advance, such as issuing an audible alarm, adjusting the seat vibration, etc. Through continuous optimization of the model, it can accurately predict the driver's behavior patterns in specific scenarios, give early warnings or take emergency treatment measures in advance, thereby improving driving safety.

[0144] Optionally, step S24 is specifically as follows:

[0145] Step S241: Extract the frequency-domain features of the driver behavior pattern data to obtain the driver behavior pattern frequency-domain feature data;

[0146] In this embodiment, when extracting the frequency-domain features of the driver behavior pattern, the driver behavior data collected by the in-vehicle sensor network (such as the steering wheel rotation angle, brake and accelerator pedal pressures, etc.) is subjected to a fast Fourier transform (FFT) to convert the time-domain data into frequency-domain data, thereby obtaining the frequency component analysis result. These frequency-domain features can help analyze the regularity and patterns of the driver's operations. For example, the high-frequency components of the frequency-domain features represent the driver's relatively frequent operations, and the low-frequency components represent the stable driving state. Through these frequency-domain data, the behavior characteristics of the driver in different driving states can be identified, further providing a basis for behavior pattern evaluation.

[0147] Step S242: Evaluate the continuity of the operation behavior and the continuity of the driving posture of the driver behavior pattern for the driver behavior pattern frequency-domain feature data to obtain the operation continuity data of the behavior pattern and the posture continuity data of the behavior pattern;

[0148] In this embodiment, when evaluating the continuity of the driving operation behavior according to the frequency-domain feature data of the driver behavior pattern, the time-series similarity of the driver's operation behavior (such as acceleration, braking, steering, etc.) is calculated to evaluate whether the operation is continuous and stable. For example, by calculating the change rate of each steering operation of the driver to determine whether the operation is continuous, and further generating the data of the continuity of the driving operation behavior pattern. Similarly, the evaluation of the continuity of the driving posture in the behavior pattern analyzes the data collected by the driver seat sensor to evaluate the stability of the posture. If it is detected that the driver often adjusts the sitting posture or changes the driving posture, the data of the continuity of the posture in the behavior pattern is relatively low, and vice versa, which can help evaluate whether the driver has unstable driving behavior or a posture that needs to be corrected.

[0149] Step S243: Evaluate the attention of the driving operation in the behavior pattern according to the data of the continuity of the driving operation behavior pattern and the data of the continuity of the driving posture, so as to obtain the data of the attention of the driving operation in the behavior pattern;

[0150] In this embodiment, when evaluating the attention of the driver's driving operation based on the data of the continuity of the driving operation behavior pattern and the data of the continuity of the driving posture, by comparing the driver's operation behavior and the continuity data of the posture with the reference value in the normal driving mode, the attention level of the driver is calculated using the weighted average method. For example, if the driver's operation behavior changes frequently and the posture is unstable, it indicates that the driver is distracted or inattentive during driving. According to these evaluation results, the system can score the driver's attention and output the corresponding attention data to help determine whether the driver needs to be reminded or warned.

[0151] Step S244: Collect the audio data in the cabin through the in-vehicle sensing network to obtain the audio data in the cabin, and perform voice activity detection according to the audio data in the cabin to obtain the voice activity data of the driver in the cabin;

[0152] In this embodiment, when the in-vehicle sensing network collects the audio data in the cabin, the in-vehicle microphone will capture the driver's voice activity in real time, and the audio data in the cabin is obtained through digital processing of the audio signal. Next, these audio signals are analyzed by the voice activity detection (VAD) algorithm to determine whether the driver is issuing a voice command or communicating with other occupants. The processing of these audio data helps to understand the speech activity and emotional state of the driver. Through voice activity detection, it is possible to identify whether the driver is in a relatively relaxed or tense state, providing a basis for subsequent attention evaluation.

[0153] Step S245: Perform statistics on the voice activity frequency and perceive the voice emotion based on the voice activity data of the driver in the cabin, so as to obtain the voice activity frequency data of the driver and the voice emotion perception data of the driver;

[0154] In this embodiment, when statistically analyzing the voice activity frequency based on the driver's voice activity data in the vehicle cabin, the number and frequency of voice commands issued by the driver per hour are statistically analyzed. By analyzing the frequency of their voice activities, the activity level of the driver within a specific period is evaluated. At the same time, emotion recognition technology (such as an emotion recognition model based on voiceprint) is used to analyze the emotional changes in the driver's voice. By analyzing features such as intonation and volume changes in the voice, emotion perception data of the driver is obtained, and then it is determined whether the driver is in an emotional state such as anxiety, tension, or relaxation. These data provide an important basis for the subsequent evaluation of the driver's language attention.

[0155] Step S246: Evaluate the driver's language attention based on the driver's voice activity frequency data and the driver's voice emotion perception data, so as to obtain the driver's language attention data;

[0156] In this embodiment, when evaluating the driver's language attention based on the driver's voice activity frequency data and the driver's voice emotion perception data, the voice activity level and emotional state of the driver are combined to evaluate their attention to the voice task. For example, if the driver has a tense tone and a high voice activity level during high-frequency voice activities, it is determined that they are facing greater driving pressure and it is inferred that their language attention is lower. Through these data, the attention level of the driver in a specific driving situation can be evaluated, assisting in the evaluation of the driver's behavior pattern.

[0157] Step S247: Perform a weighted evaluation of the driving focus of the behavior pattern based on the driver's language attention data and the driving operation attention data of the behavior pattern, so as to obtain the driver's behavior pattern focus data.

[0158] In this embodiment, when performing a weighted evaluation of the driving focus of the behavior pattern based on the driver's language attention data and the driving operation attention data of the behavior pattern, the language attention and operation attention of the driver are comprehensively analyzed. For example, by combining the language attention data and the operation attention data, if the driver's language attention and operation attention are both low, a comprehensive driver behavior pattern focus score is calculated through a weighted scoring method. If the score is lower than the preset threshold, the driver will be reminded that their attention is not concentrated, and they will be prompted to take appropriate safe driving measures.

[0159] Optionally, step S3 is specifically as follows:

[0160] Step S31: Perform Monte Carlo vehicle environment simulation based on the dynamic vehicle environment perception data, so as to obtain the dynamic vehicle environment simulation data;

[0161] In this embodiment, when performing Monte Carlo vehicle environment simulation based on dynamic vehicle environment perception data, real-time environment data from in-vehicle sensors (such as lidar, cameras, ultrasonic sensors, etc.) are first collected to obtain information such as obstacles around the vehicle, relative positions of other vehicles, and road conditions. Then, the Monte Carlo method is used to perform multiple random simulations on these data to generate different driving scenarios and environmental change situations, resulting in multiple possible vehicle environment states. The simulation data can cover traffic density changes, sudden road conditions, etc., and finally generate dynamic vehicle environment simulation data to provide diverse references for driving decisions.

[0162] Step S32: Obtain the vehicle's rated parameter set and construct a vehicle dynamics model based on the vehicle's rated parameter set;

[0163] In this embodiment, when obtaining the vehicle's rated parameter set and constructing a vehicle dynamics model based on these parameters, the basic rated parameters of the vehicle, such as maximum power, maximum torque, speed limit, acceleration, braking performance, etc., are first obtained from the vehicle control platform. By substituting these vehicle rated parameters into the dynamics model (such as physics-based dynamics equations or empirical formulas), a model that can simulate the vehicle's behavior is generated. This model can reflect the vehicle's response characteristics under different driving conditions, such as acceleration and braking behaviors at different speeds, and the impact of road conditions on vehicle performance, providing a basis for subsequent driving scenario simulations.

[0164] Step S33: Perform driver behavior prediction on the driver behavior data to be analyzed and the dynamic vehicle environment perception data through a driver behavior prediction model to obtain driver behavior prediction data;

[0165] In this embodiment, when performing driver behavior prediction on the driver behavior data to be analyzed and the dynamic vehicle environment perception data through a driver behavior prediction model, real-time driver behavior data (such as operations of the accelerator pedal, brake pedal, and steering wheel rotation angle) are first collected from the in-vehicle sensor network and combined with the dynamic vehicle environment perception data (such as road conditions, surrounding traffic conditions, etc.). The driver behavior prediction model is based on machine learning or deep learning algorithms to process these input data and infer the driver's future behavior in a specific scenario through the model. For example, predicting the driver's acceleration, deceleration, overtaking, etc. behaviors when encountering traffic congestion. Finally, the model outputs driver behavior prediction data to assist in further analyzing driving decisions.

[0166] Step S34: Perform vehicle driving scenario simulation on the driver behavior prediction data and the dynamic vehicle environment simulation data based on the vehicle dynamics model to obtain vehicle driving scenario simulation data;

[0167] In this embodiment, when simulating a vehicle driving scenario based on a vehicle dynamics model for driver behavior prediction data and dynamic vehicle environment simulation data, the driver behavior prediction data and the dynamic environment simulation data are input into the constructed vehicle dynamics model. Through simulation calculations, the driving path, speed changes, acceleration and deceleration conditions, etc. of the vehicle in a specific driving scenario are simulated. For example, the model will simulate the braking reaction of the driver when encountering an emergency stop situation, or the change in driving reaction due to environmental changes during high-speed driving. Through these simulations, the dynamic reactions of the driver and the vehicle under the action of different environmental factors can be predicted.

[0168] Step S35: Perform a driving scenario risk assessment on the vehicle driving scenario simulation data to obtain driving scenario risk assessment data.

[0169] In this embodiment, when performing a driving scenario risk assessment on the vehicle driving scenario simulation data, the potential risks in the driving scenario are evaluated based on the simulated vehicle driving data and driver reaction behaviors. For example, if the driver fails to decelerate in time during the simulation, the model will judge the possible collision risk and give corresponding risk assessment data. The risk assessment criteria can be comprehensively analyzed based on multi-dimensional factors such as safety distance, vehicle driving speed, distance to surrounding obstacles, and road conditions. Finally, the system outputs the driving scenario risk assessment data to provide real-time safety warnings or assist in decision-making for the driver.

[0170] Optionally, step S35 is specifically:

[0171] Step S351: Extract features from the vehicle driving scenario simulation data to obtain vehicle trajectory simulation data and driver behavior simulation data;

[0172] In this embodiment, when extracting features from the vehicle driving scenario simulation data, the driving path of the vehicle and the driver's operation behaviors (such as the steering wheel rotation angle, pedal pressures of the accelerator and brake pedals, etc.) are separated from the simulation data. Then, signal processing techniques (such as Fourier transform or wavelet transform) are used to perform time-domain and frequency-domain analyses on the vehicle trajectory and driver behavior. The vehicle trajectory simulation data includes path changes, speed changes, etc. of the vehicle under different driving scenarios, and the driver behavior simulation data reflects the driver's operation habits, reaction times, control accuracies, etc. under different scenarios. Through data integration and processing, the vehicle trajectory simulation data and driver behavior simulation data are obtained for subsequent analysis and evaluation.

[0173] Step S352: Statistically analyze the vehicle collision probability distribution based on the vehicle trajectory simulation data to obtain vehicle collision probability distribution data;

[0174] In this embodiment, when statistically analyzing the vehicle collision probability distribution based on the vehicle trajectory simulation data, the system first extracts the key positions (such as turning points, intersections, etc.) passed by the vehicle during the simulation, and calculates the relative distances between these positions and potential obstacles and other vehicles. Then, through probability statistical methods, the system analyzes the possible collision risks of the vehicle at each key position. By methods such as Monte Carlo simulation or Bayesian inference, the probability distribution of the vehicle collision in different driving scenarios is calculated, and the vehicle collision probability distribution data is generated. These data help to evaluate the likelihood of the vehicle facing a collision in a specific scenario and provide a basis for risk assessment.

[0175] Step S353: Evaluate the driver behavior focus of the driver behavior simulation data based on the driver behavior pattern focus data, so as to obtain the driver simulated behavior focus data;

[0176] In this embodiment, when evaluating the driver behavior focus of the driver behavior simulation data based on the driver behavior pattern focus data, the simulation data from the driver behavior prediction model is first used, including the driver's operation behaviors (such as steering, accelerating, braking) and the voice situation in the cockpit. The driver's focus data is combined with the simulated behavior data, and the driver's attention level is evaluated by analyzing the driver's reaction time, voice frequency, operation stability, and judgment accuracy in different scenarios. If the focus is low, the driver will be prompted to be alert or provided with assistance.

[0177] Step S354: Calculate the vehicle trajectory correction probability for the vehicle collision probability distribution data based on the vehicle dynamics model, so as to obtain the vehicle trajectory correction probability data;

[0178] In this embodiment, when calculating the vehicle trajectory correction probability for the vehicle collision probability distribution data based on the vehicle dynamics model, the previously generated vehicle collision probability distribution data is combined with the vehicle dynamics model to simulate the trajectory changes of the vehicle under different control strategies (such as emergency braking, steering avoidance). By calculating the correction probability of the vehicle trajectory in different scenarios, it is evaluated whether the driver's behavior can effectively avoid risks in areas with high collision risks. Specifically, when simulating the situation of the driver's emergency braking, to what extent the vehicle can reduce the collision risk, and based on this result, the probability data of the vehicle trajectory correction is calculated as the basis for optimizing driving decisions.

[0179] Step S355: Conduct a weighted assessment of the vehicle collision risk in the driving scenario for the driver simulated behavior focus data and the vehicle trajectory correction probability data, so as to obtain the driving scenario risk assessment data.

[0180] In this embodiment, when performing a weighted assessment of the vehicle collision risk in a driving scenario for the driver's simulated behavior concentration data and the vehicle trajectory correction probability data, according to the driver's behavior concentration data and the vehicle trajectory correction probability data, the collision risk of the vehicle in a specific scenario is comprehensively calculated through a weighted algorithm. If the driver's concentration is low and the probability of vehicle trajectory correction is low, a greater collision risk will be evaluated; otherwise, the risk is low. This weighted assessment result helps to determine the driver's safety in the current driving scenario and outputs the driving scenario risk assessment data, providing necessary safety warnings or intervention suggestions for the driver.

[0181] Optionally, step S4 is specifically as follows:

[0182] Step S41: Perform real-time vehicle environment perception and real-time driver behavior collection through an in-vehicle sensing network, so as to obtain real-time vehicle environment perception data and real-time driver behavior data;

[0183] In this embodiment, when performing real-time vehicle environment perception and real-time driver behavior collection through an in-vehicle sensing network, in-vehicle sensors (such as lidar, cameras, millimeter-wave radars, etc.) are used to perceive the vehicle's surrounding environment, and data including surrounding obstacles, road signs, traffic signals, etc. are collected in real time. At the same time, the driver's behavior is monitored through devices such as in-vehicle cameras, seat sensors, and steering wheel torque sensors, such as the driver's facial expressions, eye movements, hand positions, steering angles, acceleration and braking operations, etc., and the driver's behavior data is collected in real time. These data are transmitted and processed through the in-vehicle sensing network to generate real-time vehicle environment perception data and real-time driver behavior data, providing input for subsequent driving scenario integration and risk assessment.

[0184] Step S42: Integrate the real-time vehicle environment perception data and the real-time driver behavior data based on a vehicle dynamics model to obtain real-time vehicle driving scenario data;

[0185] In this embodiment, when integrating the real-time vehicle environment perception data and the real-time driver behavior data based on a vehicle dynamics model, the real-time vehicle environment perception data and the real-time driver behavior data are input into the vehicle dynamics model. The vehicle dynamics model combines the physical attributes of the vehicle (such as vehicle weight, vehicle speed, steering angle, tire friction coefficient, etc.) and external environment information (such as road conditions, obstacle distance, traffic signal status, etc.), and simulates the dynamic response of the vehicle. By integrating the real-time vehicle environment perception data and the driver behavior data, the model generates a complete set of vehicle driving scenario data, reflecting the actual state of the vehicle in the current environment, including the vehicle's movement trajectory, the driver's control behavior, and potential environmental risks.

[0186] Step S43: Calculate the scenario similarity based on the vehicle driving scenario simulation data and the real-time vehicle driving scenario data, so as to obtain the driving scenario similarity data;

[0187] In this embodiment, when calculating the scenario similarity based on the vehicle driving scenario simulation data and the real-time vehicle driving scenario data, different driving scenario templates are created using historical data and simulation data. These templates include different driving environments, road conditions, and driver behavior patterns. Then, by calculating the similarity between the real-time vehicle driving scenario data and the known scenario templates, a multi-dimensional distance metric method (such as Euclidean distance, Manhattan distance, or cosine similarity, etc.) is used to compare the real-time driving scenario with the simulated scenario. The calculated scenario similarity data can reflect the similarity between the current driving situation and the historical driving situation, helping the system to judge the potential risks and processing strategies of the current situation.

[0188] Step S44: Extract the vehicle warning trigger parameters from the vehicle rated parameter group, so as to obtain the vehicle warning trigger parameter group, and allocate the risk level vehicle warning instructions to the vehicle warning trigger parameter group according to the driving scenario risk assessment data, so as to obtain the multi-risk level warning trigger parameter group;

[0189] In this embodiment, when extracting the vehicle warning trigger parameters from the vehicle rated parameter group, by analyzing the basic parameters of the vehicle (such as maximum speed, acceleration ability, braking distance, etc.) and the real-time state of the vehicle (such as tire temperature, fuel quantity, power, driver behavior pattern, etc.), a parameter group related to warning trigger is extracted. According to the driving scenario risk assessment data, the warning trigger parameter group is allocated to different risk levels according to the preset risk levels. For example, in the highway driving scenario, if it is found that the driver is inattentive or has abnormal driving behavior, the driving scenario risk is relatively high, and a warning instruction with a higher risk level is selected, such as increasing the voice alarm or triggering the seat vibration feedback. In the urban driving situation, a low-risk level driving scenario only triggers a mild warning, such as a voice reminder.

[0190] Step S45: Use the driving scenario similarity data and perform a real-time driving scenario risk assessment on the real-time vehicle driving scenario data based on the driving scenario risk assessment data, so as to obtain the real-time driving scenario risk assessment data;

[0191] In this embodiment, when using the driving scenario similarity data and performing real-time driving scenario risk assessment on the real-time vehicle driving scenario data based on the driving scenario risk assessment data, the risk in the current driving situation and past similar situations is evaluated by combining the real-time driving scenario data with the scenario similarity data. Comprehensive evaluation will be carried out according to the driver's behavior patterns (such as fatigue driving, distracted driving), environmental risks (such as road icing, driving in rainy days), and the current state of the vehicle (such as braking distance, vehicle speed, etc.). If a high risk level is detected, the real-time driving scenario risk assessment data will reflect this risk and guide subsequent warning and intervention strategies. The real-time driving scenario data will be compared with the historical scenarios obtained through similarity calculation before. Suppose it is recognized that the current scenario is very similar to a certain past dangerous driving scenario (such as ignoring the traffic conditions ahead when driving at high speed), and the historical data indicates that this scenario has a high accident risk. At this time, a risk assessment model constructed based on the driving scenario risk assessment data (a driving risk assessment model constructed and trained by using machine learning algorithms (such as decision tree, logistic regression, etc.) with the driving scenario risk assessment data and historical driving scenario data as inputs) will analyze the driver's behavior and the current vehicle conditions (such as braking distance, vehicle speed, environmental conditions, etc.) to generate real-time driving scenario risk assessment data. If the risk assessment data indicates that the current scenario is highly dangerous, a warning will be triggered and data support will be provided for the next intervention.

[0192] Step S46: Associate warning trigger instructions with the multi-risk-level warning trigger parameter group through a predefined warning instruction set, so as to obtain a multi-level warning trigger instruction set;

[0193] In this embodiment, when associating warning trigger instructions with the multi-risk-level warning trigger parameter group through a predefined warning instruction set, the system first predefined different warning instruction sets according to multiple risk levels (such as low, medium, and high risks), which include various feedback means such as visual, auditory, and tactile. According to the driving scenario risk assessment data and the multi-risk-level warning trigger parameter group, the risk level is matched with the corresponding warning instruction. For example, in the case of low risk, the driver will only be reminded by voice, while in the case of high risk, not only an emergency warning will be issued, but also intervention will be carried out through seat vibration, brake assist, etc.

[0194] Step S47: Match warning instructions with the real-time driving scenario risk assessment data based on the multi-level warning trigger instruction set, so as to obtain a vehicle warning instruction set and transmit it to the vehicle control platform to execute the vehicle driving warning task.

[0195] In this embodiment, when matching warning instructions for real-time driving scenario risk assessment data based on a multi-level warning trigger instruction set, hierarchical matching will be performed according to the real-time driving scenario risk assessment data. If the risk assessment data indicates a high collision or safety hazard in the current scenario, the most suitable high-risk instructions (such as emergency braking instructions, acceleration warnings, steering wheel vibrations, etc.) will be selected from the warning instruction set, and relevant operations will be executed through the vehicle control platform to timely intervene in the driving behavior and reduce potential dangers. These warning instructions will be transmitted to the vehicle's execution system to ensure timely responses in dangerous situations.

[0196] Optionally, step S6 is specifically as follows:

[0197] Step S61: Calculate the driving scenario similarity based on the vehicle driving scenario simulation data and the real-time driving warning scenario simulation data, so as to obtain the real-time driving warning scenario similarity data;

[0198] In this embodiment, the driving scenario similarity is calculated based on the vehicle driving scenario simulation data and the real-time driving warning scenario simulation data, so as to obtain the real-time driving warning scenario similarity data. The vehicle driving scenario simulation data is compared with the real-time driving warning scenario simulation data. The real-time driving warning scenario simulation data comes from the vehicle's sensor inputs, including environmental perception (such as road conditions, weather conditions, etc.) and driver behavior data. The similarity calculation method uses the Euclidean distance or cosine similarity algorithm. By calculating the differences between the two sets of data, the real-time driving warning scenario similarity data is obtained, which reflects the similarity between the current driving scenario and the historical warning scenarios.

[0199] Step S62: Use the real-time driving warning scenario similarity data and, based on the driving scenario risk assessment data, conduct a real-time driving warning scenario risk assessment on the real-time driving warning scenario simulation data, so as to obtain the real-time driving warning scenario risk assessment data;

[0200] In this embodiment, the real-time driving warning scenario similarity data is used, and a real-time driving warning scenario risk assessment is conducted on the real-time driving warning scenario simulation data based on the driving scenario risk assessment data, so as to obtain the real-time driving warning scenario risk assessment data. At this time, based on historical data and a driving scenario risk assessment model (for example, a decision tree model or a deep neural network model), combined with the real-time driving warning scenario similarity data, the risk level of the current driving scenario can be evaluated. The model will analyze the driver's driving behavior (such as sudden braking, steering, etc.) and environmental data (such as slippery roads, haze, etc.), and combined with the scenario similarity data, output the real-time driving warning scenario risk assessment data. If the similarity is high and the environmental risk is large, a high risk level will be evaluated and used as a basis for subsequent intervention.

[0201] Step S63: Perform a risk assessment comparison on the real-time driving scenario risk assessment data and the real-time driving warning scenario risk assessment data to obtain risk assessment error data;

[0202] In this embodiment, a risk assessment comparison is performed on the real-time driving scenario risk assessment data and the real-time driving warning scenario risk assessment data to obtain risk assessment error data. The real-time driving scenario risk assessment data is compared with the real-time driving warning scenario risk assessment data to analyze the differences between the two. Specifically, the risk assessment error data reflects the deviation between the risk assessments of the two scenarios. For example, if the result of the real-time driving scenario risk assessment is relatively optimistic, while the result of the real-time driving warning scenario risk assessment shows a higher risk, an error will be calculated.

[0203] Step S64: Perform a driver behavior focus assessment on the real-time warning driver behavior data and the real-time warning vehicle environment perception data based on the driver behavior pattern focus data to obtain real-time driver behavior focus data;

[0204] In this embodiment, a driver behavior focus assessment is performed on the real-time warning driver behavior data and the real-time warning vehicle environment perception data based on the driver behavior pattern focus data to obtain real-time driver behavior focus data. In this step, the real-time warning driver behavior data is analyzed, including the driver's operations (such as braking, accelerating, steering), as well as facial expressions, eye movement data, in-vehicle voice, etc., to evaluate whether the driver is concentrating. If the driver behavior data indicates that the driver shows fatigue, distraction, or inattentiveness, the driver behavior focus assessment data will be further increased, and the driver's focus level will be weighted and evaluated based on the real-time vehicle environment perception data (such as road conditions, traffic conditions).

[0205] Step S65: Perform a weighted assessment of the vehicle warning feedback degree based on the risk assessment error data and the real-time driver behavior focus data to obtain vehicle warning feedback degree data;

[0206] In this embodiment, a weighted assessment of the vehicle warning feedback degree is performed based on the risk assessment error data and the real-time driver behavior focus data to obtain vehicle warning feedback degree data. In this step, the system combines multiple weight strategies to perform a weighted assessment of the vehicle warning feedback degree according to the risk assessment error data (i.e., the magnitude of the assessment error) and the driver's behavior focus data. If the driver's focus is low and there is a large risk assessment error, the vehicle warning feedback degree will be low; otherwise, the vehicle warning feedback degree will be high. The weighted vehicle warning feedback degree data is used to trigger appropriate warning strategies.

[0207] Step S66: Select emergency intervention instructions from the multi-level warning trigger instruction set based on the vehicle warning feedback degree data, so as to obtain an emergency intervention decision execution instruction set and transmit it to the vehicle control platform to execute the vehicle driving control task.

[0208] In this embodiment, emergency intervention instructions are selected from the multi-level warning trigger instruction set based on the vehicle warning feedback degree data, so as to obtain an emergency intervention decision execution instruction set and transmit it to the vehicle control platform to execute the vehicle driving control task. By analyzing the vehicle warning feedback degree data, the urgency of the current situation is judged, and appropriate intervention measures are selected from the multi-level warning trigger instruction set. The intervention measures include sound alarms, seat vibrations, automatic braking, speed limits, etc. Based on the assessment of the risk level and the driver behavior pattern focus, the most appropriate emergency response will be determined. If the vehicle warning feedback degree is low, the selected emergency intervention instruction will be higher than the risk level of the vehicle warning instruction set. The selected intervention instruction is then transmitted to the vehicle control platform to execute the corresponding vehicle driving control task to ensure safe driving.

[0209] Optionally, this specification also provides a data model processing system for executing the data model processing method described above. The data model processing system includes:

[0210] A vehicle environment perception module, configured to obtain multi-modal vehicle-borne sensing data, perform multi-modal sensing data fusion according to the multi-modal vehicle-borne sensing data to obtain a vehicle-borne sensing network; perform vehicle environment perception according to the vehicle-borne sensing network to obtain dynamic vehicle environment perception data;

[0211] A driver behavior pattern modeling module, configured to collect driver behavior data through the vehicle-borne sensing network to obtain driver behavior data, perform driver behavior pattern recognition according to the driver behavior data to obtain driver behavior pattern data; perform behavior pattern modeling on the driver behavior pattern data to obtain a driver behavior prediction model;

[0212] A driving scenario risk assessment module, configured to perform vehicle driving scenario simulation based on the driver behavior prediction model and the dynamic vehicle environment perception data to obtain vehicle driving scenario simulation data, and perform driving scenario risk assessment on the vehicle driving scenario simulation data to obtain driving scenario risk assessment data;

[0213] A warning strategy analysis module, configured to perform vehicle multi-level warning strategy analysis according to the driving scenario risk assessment data to obtain a vehicle warning instruction set and transmit it to the vehicle control platform to execute the vehicle driving warning task;

[0214] The driver behavior prediction module is used to collect real-time driver behavior data and perceive the real-time vehicle environment through the in-vehicle sensing network, so as to obtain real-time warning driver behavior data and real-time warning vehicle environment perception data, and simulate the vehicle driving warning scenario for the real-time warning driver behavior data and real-time warning vehicle environment perception data through the driver behavior prediction model, so as to obtain real-time driving warning scenario simulation data;

[0215] The warning feedback degree evaluation module is used to evaluate the vehicle warning feedback degree for the real-time driving warning scenario simulation data, so as to obtain vehicle warning feedback degree data; automatically make decisions on vehicle emergency intervention measures according to the vehicle warning feedback degree data, so as to obtain an emergency intervention decision execution instruction set, and transmit it to the vehicle control platform to execute the vehicle driving control task.

Claims

1. A data model processing method, characterized in that: The following steps are involved: Step S1: acquiring multimodal vehicle-mounted sensor data, and performing multimodal sensor data fusion according to the multimodal vehicle-mounted sensor data, thereby obtaining a vehicle-mounted sensor network; Carry out vehicle environment perception based on the vehicle-mounted sensor network to obtain dynamic vehicle environment perception data; Step S2: collecting driver behavior data through the vehicle-mounted sensor network to obtain driver behavior data, and performing driver behavior pattern recognition based on the driver behavior data to obtain driver behavior pattern data; Conducting behavior pattern modeling on driver behavior pattern data to obtain a driver behavior prediction model; Step S3: performing a vehicle driving scenario simulation based on the driver behavior prediction model and the dynamic vehicle environment perception data to obtain vehicle driving scenario simulation data, and performing a driving scenario risk assessment on the vehicle driving scenario simulation data to obtain driving scenario risk assessment data; Step S4: Performing a vehicle multi-level warning strategy analysis based on the driving scenario risk assessment data, thereby obtaining a vehicle warning instruction set, and transmitting it to the vehicle control platform to execute the vehicle driving warning task; Step S5: collecting real-time driver behavior data and real-time vehicle environment perception through the vehicle-mounted sensor network, thereby obtaining real-time warning driver behavior data and real-time warning vehicle environment perception data, and performing driver behavior prediction on the real-time warning driver behavior data and real-time warning vehicle environment perception data through a driver behavior prediction model, thereby obtaining real-time warning driver behavior prediction data; Step S6: evaluating the vehicle warning feedback degree of the real-time driving warning scenario simulation data, thereby obtaining vehicle warning feedback degree data; Automatic decisions on vehicle emergency intervention measures are made based on the vehicle warning feedback data, thereby obtaining an emergency intervention decision execution instruction set and transmitting it to the vehicle control platform to execute vehicle driving control tasks.

2. The data model processing method according to claim 1, characterized in that: Step S1 is specifically as follows: Step S11: acquiring multimodal vehicle-mounted sensor data through a preset vehicle-mounted sensor group, and performing data preprocessing on the multimodal vehicle-mounted sensor data, thereby obtaining multimodal vehicle-mounted sensor data to be analyzed; Step S12: performing spatial classification of the multimodal vehicle-mounted sensor data to be analyzed, thereby obtaining multimodal sensor data inside the vehicle cabin and multimodal sensor data outside the vehicle cabin; Step S13: performing sensor data fusion on the multimodal sensor data inside the vehicle cabin and the multimodal sensor data outside the vehicle cabin, so as to obtain the fused sensor data inside the vehicle cabin and the fused sensor data outside the vehicle cabin; Step S14: constructing a vehicle sensor network space according to the in-cabin fused sensor data and the out-cabin fused sensor data, thereby obtaining an in-vehicle sensor network; Step S15: Perform vehicle environment perception based on the vehicle-mounted sensor network to obtain dynamic vehicle environment perception data.

3. The data model processing method according to claim 2, characterized in that: Step S13 is specifically as follows: Extracting time-frequency features of multi-modal sensing data in the vehicle cabin, thereby obtaining time-frequency feature data of multi-modal sensing in the vehicle cabin; Identify the ambient noise in the cabin based on the multi-modal sensing time-frequency characteristic data in the cabin, so as to obtain the ambient noise data in the cabin; Performing environmental noise filtering on the multi-modal sensor data in the cabin according to the environmental noise data in the cabin, so as to obtain the multi-modal sensor data in the cabin to be fused; Perform principal component feature correlation on the multi-modal sensor data to be fused in the vehicle cabin, so as to obtain the fused sensor data in the vehicle cabin; Performing multi-sensor data alignment on the multi-modal sensor data outside the vehicle cabin, thereby obtaining aligned multi-modal sensor data outside the vehicle cabin; Performing statistics on the spatial distribution of the sensors outside the vehicle cabin according to the aligned multi-modal sensor data outside the vehicle cabin, thereby obtaining the spatial distribution data of the sensors outside the vehicle cabin, and constructing a unified spatial coordinate system of the vehicle based on the spatial distribution data of the sensors outside the vehicle cabin; The multi-modal sensor data outside the vehicle cabin is aligned in a unified spatial coordinate system and then spatially fused and interpolated in a unified scale to obtain the fused sensor data outside the vehicle cabin.

4. The data model processing method according to claim 1, characterized in that: Step S2 is specifically as follows: Step S21: collecting driver behavior data through the vehicle-mounted sensor network to obtain driver behavior data, and performing data preprocessing on the driver behavior data to obtain driver behavior data to be analyzed; Step S22: extracting driver behavior features according to the driver behavior data to be analyzed, thereby obtaining driver operation behavior feature data and driver behavior posture feature data; Step S23: performing driving behavior pattern recognition based on the driver's operation behavior feature data and the driver's behavior posture feature data, thereby obtaining driver behavior pattern data; Step S24: evaluating the driver behavior pattern concentration on the driver behavior pattern data, thereby obtaining the driver behavior pattern concentration data; Step S25: performing time-series correlation on the dynamic vehicle environment perception data and the driver behavior pattern data, thereby obtaining behavior pattern-vehicle environment correlation data; Step S26: constructing a driver behavior prediction model based on the behavior pattern-vehicle environment association data, the driver behavior pattern data, and the driver behavior pattern concentration data.

5. The data model processing method according to claim 4, characterized in that: Step S24 is specifically as follows: Step S241: extracting frequency domain features of the driver behavior pattern data, thereby obtaining frequency domain feature data of the driver behavior pattern; Step S242: performing a behavior pattern operation behavior continuity evaluation and a behavior pattern driving posture continuity evaluation on the driver behavior pattern frequency domain feature data, thereby obtaining behavior pattern operation continuity data and behavior pattern posture continuity data; Step S243: performing a behavior pattern driving operation attention evaluation according to the behavior pattern operation continuity data and the behavior pattern posture continuity data, thereby obtaining the behavior pattern driving operation attention data; Step S244: collecting in-cabin audio data through the vehicle-mounted sensor network to obtain in-cabin audio data, and performing voice activity detection based on the in-cabin audio data to obtain in-cabin driver voice activity data; Step S245: performing voice activity frequency statistics and voice emotion perception based on the driver's voice activity data in the vehicle cabin, thereby obtaining the driver's voice activity frequency data and the driver's voice emotion perception data; Step S246: evaluating the driver's speech attention according to the driver's speech activity frequency data and the driver's speech emotion perception data, thereby obtaining the driver's speech attention data; Step S247: Perform a behavior pattern driving concentration weighted evaluation based on the driver's language attention data and the behavior pattern driving operation attention data, so as to obtain the driver's behavior pattern concentration data.

6. The data model processing method according to claim 1, characterized in that: Step S3 is specifically as follows: Step S31: performing Monte Carlo vehicle environment simulation according to the dynamic vehicle environment perception data, thereby obtaining dynamic vehicle environment simulation data; Step S32: obtaining a vehicle rated parameter group, and constructing a vehicle dynamics model based on the vehicle rated parameter group; Step S33: performing driver behavior prediction on the driver behavior data to be analyzed and the dynamic vehicle environment perception data through a driver behavior prediction model, thereby obtaining driver behavior prediction data; Step S34: performing vehicle driving scenario simulation on the driver behavior prediction data and the dynamic vehicle environment simulation data based on the vehicle dynamics model, thereby obtaining vehicle driving scenario simulation data; Step S35: Perform driving scenario risk assessment on the vehicle driving scenario simulation data to obtain driving scenario risk assessment data.

7. The data model processing method according to claim 6, characterized in that: Step S35 is specifically as follows: Step S351: extracting features from the vehicle driving scenario simulation data, thereby obtaining vehicle trajectory simulation data and driver behavior simulation data; Step S352: performing vehicle collision probability distribution statistics according to the vehicle trajectory simulation data, thereby obtaining vehicle collision probability distribution data; Step S353: evaluating the driver behavior concentration of the driver behavior simulation data according to the driver behavior pattern concentration data, thereby obtaining the driver simulation behavior concentration data; Step S354: calculating the vehicle trajectory correction probability based on the vehicle collision probability distribution data based on the vehicle dynamics model, thereby obtaining vehicle trajectory correction probability data; Step S355: Perform a driving scenario vehicle collision risk weighted assessment on the driver simulation behavior concentration data and the vehicle trajectory correction probability data, thereby obtaining driving scenario risk assessment data.

8. The data model processing method according to claim 1, characterized in that: Step S4 is specifically as follows: Step S41: Real-time vehicle environment perception and real-time driver behavior collection are performed through the vehicle-mounted sensor network, thereby obtaining real-time vehicle environment perception data and real-time driver behavior data; Step S42: integrating the real-time vehicle environment perception data and the real-time driver behavior data into a vehicle driving scenario based on a vehicle dynamics model, thereby obtaining real-time vehicle driving scenario data; Step S43: Calculating the scenario similarity based on the vehicle driving scenario simulation data and the real-time vehicle driving scenario data, thereby obtaining driving scenario similarity data; Step S44: extracting vehicle warning trigger parameters from the vehicle rated parameter group to obtain a vehicle warning trigger parameter group, and assigning risk level vehicle warning instructions to the vehicle warning trigger parameter group according to the driving scenario risk assessment data to obtain a multi-risk level warning trigger parameter group; Step S45: using the driving scenario similarity data and based on the driving scenario risk assessment data, performing a real-time driving scenario risk assessment on the real-time vehicle driving scenario data, thereby obtaining real-time driving scenario risk assessment data; Step S46: Associating the warning triggering instructions with the warning triggering parameter groups of multiple risk levels through the predefined warning instruction set, thereby obtaining a multi-level warning triggering instruction set; Step S47: Match the warning instructions to the real-time driving scenario risk assessment data based on the multi-level warning trigger instruction set, thereby obtaining a vehicle warning instruction set, and transmitting it to the vehicle control platform to execute the vehicle driving warning task.

9. The data model processing method according to claim 1, characterized in that: Step S6 is specifically as follows: Step S61: Calculating driving scenario similarity based on vehicle driving scenario simulation data and real-time driving warning scenario simulation data, thereby obtaining real-time driving warning scenario similarity data; Step S62: using the real-time driving warning scenario similarity data and based on the driving scenario risk assessment data, performing a real-time driving warning scenario risk assessment on the real-time driving warning scenario simulation data, thereby obtaining real-time driving warning scenario risk assessment data; Step S63: performing risk assessment comparison on the real-time driving scenario risk assessment data and the real-time driving warning scenario risk assessment data, thereby obtaining risk assessment error data; Step S64: evaluating the driver behavior concentration of the real-time warning driver behavior data and the real-time warning vehicle environment perception data according to the driver behavior pattern concentration data, thereby obtaining real-time driver behavior concentration data; Step S65: Performing a weighted evaluation of the vehicle warning feedback degree based on the risk assessment error data and the real-time driver behavior concentration data, thereby obtaining vehicle warning feedback degree data; Step S66: Select emergency intervention instructions for the multi-level warning trigger instruction set according to the vehicle warning feedback data, so as to obtain an emergency intervention decision execution instruction set, and transmit it to the vehicle control platform to execute the vehicle driving control task.

10. A data model processing system, characterized in that: Used to execute the data model processing method according to claim 1, the data model processing system comprises: The vehicle environment perception module is used to obtain multi-modal vehicle-mounted sensor data, and perform multi-modal sensor data fusion based on the multi-modal vehicle-mounted sensor data to obtain a vehicle-mounted sensor network; perform vehicle environment perception based on the vehicle-mounted sensor network to obtain dynamic vehicle environment perception data; A driver behavior pattern modeling module is used to collect driver behavior data through an on-vehicle sensor network to obtain driver behavior data, and to identify driver behavior patterns based on the driver behavior data to obtain driver behavior pattern data; and to perform behavior pattern modeling on the driver behavior pattern data to obtain a driver behavior prediction model; A driving scenario risk assessment module, which is used to simulate a vehicle driving scenario based on a driver behavior prediction model and dynamic vehicle environment perception data, thereby obtaining vehicle driving scenario simulation data, and to perform a driving scenario risk assessment on the vehicle driving scenario simulation data, thereby obtaining driving scenario risk assessment data; The warning strategy analysis module is used to perform vehicle multi-level warning strategy analysis based on driving scenario risk assessment data, thereby obtaining a vehicle warning instruction set and transmitting it to the vehicle control platform to execute the vehicle driving warning task; The driver behavior prediction module is used to collect real-time driver behavior data and real-time vehicle environment perception through the vehicle-mounted sensor network, so as to obtain real-time warning driver behavior data and real-time warning vehicle environment perception data, and simulate vehicle driving warning scenarios on the real-time warning driver behavior data and real-time warning vehicle environment perception data through the driver behavior prediction model, so as to obtain real-time driving warning scenario simulation data; The warning feedback evaluation module is used to evaluate the vehicle warning feedback degree of the real-time driving warning scenario simulation data, so as to obtain the vehicle warning feedback degree data; according to the vehicle warning feedback degree data, the vehicle emergency intervention measures are automatically decided, so as to obtain the emergency intervention decision execution instruction set, and transmit it to the vehicle control platform to execute the vehicle driving control task.

Images