Auxiliary driving method and device, equipment and medium

Abstract

The invention discloses an auxiliary driving method. The method comprises the following steps: acquiring a driving strategy association factor; performing feature extraction on the driving strategy correlation factors to obtain driving strategy correlation features; dynamically distributing the weights of the plurality of driving strategy associated features; performing weighted fusion on the plurality of driving strategy associated features to obtain a driving strategy fusion feature; determining a risk level evaluation index based on the driving strategy fusion feature, and obtaining a risk evaluation result based on the risk level evaluation index and an index threshold; the index threshold value is generated based on the personalized data of the driver and the traffic environment data; and generating an auxiliary driving strategy according to a risk assessment result. According to the method, the multi-dimensional driver data is fused, the feature weight is dynamically distributed, and the risk assessment result is generated in combination with the personalized index threshold, so that the problem of misjudgment caused by single data dimension and fixed weight of a traditional system is solved, and the method has the advantages of improving the driving risk assessment accuracy, enhancing the system adaptability and improving the driving safety.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving safety technology, and specifically to an assisted driving method, device, equipment and medium. Background Art

[0002] Ensuring driving safety has become a critical issue in the development of intelligent transportation systems. To ensure this, related technologies utilize multi-source data and assign fixed weights to each source to monitor the driver's status. This fixed-weight approach to integrating multi-source features is inadequate for dynamic driving scenarios, preventing intelligent driving systems from responding appropriately in emergencies and compromising driving safety. Summary of the Invention

[0003] In view of the above-mentioned shortcomings of the prior art, the present application provides an assisted driving method, device, equipment and medium to solve at least one defect in the prior art.

[0004] To achieve the above and other purposes, the present application provides an assisted driving method, which includes: Acquiring a plurality of driving strategy-related factors, the plurality of driving strategy-related factors comprising at least two of the driver's state data, the driver's physiological data, and the driver's irrational state data; Extract features of multiple driving strategy related factors to obtain multiple driving strategy related features; Dynamically assign weights to multiple driving strategy-related features; performing weighted fusion on the multiple driving strategy associated features based on the weights of the multiple driving strategy associated features to obtain a driving strategy fusion feature; Determining a risk level assessment index based on the driving strategy fusion feature, and obtaining a risk assessment result based on the risk level assessment index and an index threshold; wherein the index threshold is generated based on the driver's personalized data and traffic environment data; An assisted driving strategy is generated based on the risk assessment results.

[0005] In one embodiment of the present application, the personalized data includes at least one of the following: driver's physical health status data, driver's driving habit data, and driver's individual attribute data.

[0006] In one embodiment of the present application, the risk assessment result includes a risk assessment level and a confidence level corresponding to the risk assessment level; and the assisted driving method further includes: Performing a first verification on the risk assessment result based on the confidence level and the confidence threshold; When the first verification is passed, the risk assessment result is verified for the second time based on a preset redundancy rule.

[0007] In one embodiment of the present application, the risk level assessment indicator includes at least one of the following: Attention distraction index, heart rate variability, pupil dilation duration, respiratory rate, steering wheel grip entropy, and cardiac arrest time.

[0008] In one embodiment of the present application, the multiple driving strategy association factors include a first driving strategy association factor stored locally and a second driving strategy association factor stored in the cloud. Obtaining the multiple driving strategy association factors includes: fusing the first driving strategy association factor and the second driving strategy association factor based on a federated learning method to obtain the multiple driving strategy association factors.

[0009] In one embodiment of the present application, the dynamic allocation of weights of multiple driving strategy-related features includes: Obtain real-time environmental features and real-time vehicle status features; Obtain a reliability score based on real-time environmental features, real-time vehicle status features, and a trained reliability prediction model; Determining a weight for each driving strategy-associated feature based on the reliability score to obtain a dynamic weight matrix; The weights of the plurality of driving strategy-related features are dynamically allocated based on the dynamic weight matrix.

[0010] In one embodiment of the present application, the driving strategy fusion features include short-term features, mid-term features, and long-term features.

[0011] To achieve the above-mentioned and other related purposes, the present application provides a driving assistance device, comprising: a data acquisition module, configured to acquire a plurality of driving strategy-related factors, wherein the plurality of driving strategy-related factors include at least two of the driver's state data, the driver's physiological data, and the driver's irrational state data; A feature extraction module is used to extract features of multiple driving strategy related factors to obtain multiple driving strategy related features; A weight allocation module is used to dynamically allocate weights of multiple driving strategy-related features; a feature fusion module, configured to perform weighted fusion on the plurality of driving strategy-related features based on the weights of the plurality of driving strategy-related features to obtain a driving strategy fusion feature; a risk assessment module, configured to determine a risk level assessment index based on the driving strategy fusion feature, and obtain a risk assessment result based on the risk level assessment index and an index threshold; wherein the index threshold is generated based on the driver's personalized data and traffic environment data; A strategy generation module is used to generate an assisted driving strategy based on the risk assessment results.

[0012] To achieve the above-mentioned and other related purposes, the present application provides a device, comprising: one or more processors; and The memory is configured to store one or more programs, and when the one or more programs are executed by the one or more processors, the memory implements the method described above.

[0013] To achieve the above objectives and other related objectives, the present application provides one or more machine-readable media having instructions stored thereon, which, when executed by one or more processors, enable the processors to perform the described method.

[0014] Beneficial effects of this application: An assisted driving method of the present application includes: obtaining multiple driving strategy related factors, wherein the multiple driving strategy related factors include at least two of the driver's state data, the driver's physiological data, and the driver's irrational state data; performing feature extraction on the multiple driving strategy related factors to obtain multiple driving strategy related features; dynamically assigning weights to the multiple driving strategy related features; performing weighted fusion of the multiple driving strategy related features based on the weights of the multiple driving strategy related features to obtain a driving strategy fusion feature; determining a risk level assessment index based on the driving strategy fusion feature, and obtaining a risk assessment result based on the risk level assessment index and an index threshold; wherein the index threshold is generated based on the driver's personalized data and traffic environment data; and generating an assisted driving strategy based on the risk assessment result. The present application solves the problem of misjudgment caused by the single data dimension and fixed weight of the traditional system by fusing multi-dimensional driver data and dynamically assigning feature weights, and combining personalized index thresholds to generate risk assessment results. It has the advantages of improving the accuracy of driving risk assessment, enhancing system adaptability, and improving driving safety.

[0015] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are incorporated into and constitute a part of the specification, illustrating embodiments consistent with the present application and, together with the specification, serving to explain the principles of the present application. It is obvious that the drawings described below are merely some embodiments of the present application, and a person of ordinary skill in the art can derive other drawings based on these drawings without inventive effort. In the drawings: Figure 1 This is a flowchart of an assisted driving method according to an embodiment of the present application; Figure 2 This is a flowchart of dynamically allocating weights of multiple driving strategy-related features according to an embodiment of the present application; Figure 3 This is a functional block diagram of a driving assistance device according to an embodiment of the present application; Figure 4 A schematic diagram of the structure of a computer system suitable for implementing the memory of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0017] The following describes the embodiments of the present application through specific examples. Those skilled in the art can easily understand the other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0018] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application. Therefore, the illustrations only show components related to the present application and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.

[0019] Although the terms "first," "second," "A," and "B," etc. may be used herein to describe various elements, these elements should not be limited by these terms and are merely used to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the technology described below. The term "and / or" includes a combination of a plurality of related items or any of the plurality of related items.

[0020] As used herein, unless the context indicates otherwise, the singular form is intended to include the plural form, and it will be understood that the term "comprising" means the presence of stated features, quantities, steps, operations, elements, or combinations thereof, but does not preclude the presence or addition of one or more other features, quantities, steps, operations, elements, components, or combinations thereof.

[0021] Before describing the components in detail, it is intended to clarify that the components in this specification are divided only by the primary function of each component. That is, two or more components described below may be combined into one component, or may be divided into two or more components based on more detailed functions. In addition to the primary function of the component, each component described below may also perform some or all of the functions of other components, and some of the primary functions of each component may be exclusively performed by other components.

[0022] Before explaining this application in detail, each term is explained first.

[0023] Personalized baseline data refers to a basic reference standard set based on individual characteristics, needs, or behavior patterns. For example, in the health field, a personalized baseline can refer to a range of health indicators based on age, physical condition, and lifestyle habits.

[0024] The embodiments of the present application respectively propose an assisted driving method, an assisted driving device, an assisted driving equipment, and a computer-readable storage medium, and these embodiments will be described in detail below.

[0025] See also Figure 1 , Figure 1 This is a flow chart of an assisted driving method according to an embodiment of the present application. Figure 1 As shown, the assisted driving method includes at least steps S110 to S140: Step S110, obtaining a plurality of driving strategy-related factors, where the plurality of driving strategy-related factors include at least two of the driver's state data, the driver's physiological data, and the driver's irrational state data; The multiple driving strategy related factors may include at least two of the driver's state data, the driver's physiological data, the driver's irrational state data, etc. For example, including state data and physiological data, or state data and irrational state data, or physiological data and irrational state data, or state data, physiological data and irrational state data The driver's status data can be collected through the visual perception unit. The visual perception unit includes an infrared camera and a 3D ToF sensor. The infrared camera is installed on top of the dashboard behind the steering wheel, facing vertically toward the driver's face. It supports 840nm wavelength infrared illumination to ensure pupil tracking accuracy in low-light / nighttime environments. The 3D ToF sensor is integrated on the back of the interior rearview mirror and generates three-dimensional point cloud data of the driver's head (resolution 640×480). This is used to calculate the head deflection angle (±30° range) and nodding frequency. Of course, the visual perception unit can use other devices besides infrared cameras and 3D ToF sensors, and there are no restrictions here, as long as they can achieve the collection of status data. The infrared camera and 3D ToF sensor can also be installed elsewhere in the vehicle, and there are no restrictions here.

[0026] The driver's physiological data can be collected through a physiological monitoring unit. This unit includes a steering wheel grip force sensor and a millimeter-wave bioradar. The steering wheel grip force sensor uses a flexible capacitive pressure array (16×16 grid) embedded at the 3 and 9 o'clock positions on the steering wheel to generate a real-time grip force distribution heat map, detecting muscle relaxation or abnormal grip (such as epileptic seizures). The millimeter-wave bioradar, installed in the recessed area of ​​the front reading light on the roof, operates at a frequency of 60GHz and uses beamforming technology to target the driver's chest area, extracting heart rate (HR), respiratory rate (RR), and body activity magnitude (BAM) signals. Of course, the physiological monitoring unit can utilize other devices besides the steering wheel grip force sensor and millimeter-wave bioradar, without any restrictions, as long as they can collect physiological data. The steering wheel grip force sensor and millimeter-wave bioradar can also be installed elsewhere in the vehicle, without any restrictions.

[0027] Among them, the driver's irrational state data can be collected through the behavior perception unit. The behavior perception unit may include: a pedal pressure sensor. Pedal pressure sensor: The accelerator / brake pedal has a built-in piezoelectric film to record the operation frequency and force curve, and identify irrational stepping (such as long-term non-release or high-frequency jitter); the behavior perception unit may also include: a multi-directional microphone array in the car, distributed on the A-pillar and the roof console, combined with a noise reduction algorithm to extract the driver's voice, analyze the speech speed, tone and semantic logic anomalies (such as intermittent, repetitive). Of course, the behavior perception unit can use other devices besides the pedal pressure sensor and the multi-directional microphone array in the car, and there is no restriction here, as long as it can achieve the collection of irrational state data.

[0028] Step S120, extracting features from the multiple driving strategy related factors to obtain multiple driving strategy related features; By processing the collected driving strategy-related factors, at least two of the driver's state data, physiological data, and irrational state data are input into a driving risk assessment model (a prediction model based on multimodal data fusion, which is obtained through historical data training). The feature extraction network of the driving risk assessment model is used to construct a multivariate feature set including state features, biological features, and irrational state features based on deep feature engineering, thereby establishing a multidimensional feature space to accurately characterize the generation mechanism of driving strategies.

[0029] To extract state features, we first use an infrared camera to obtain a facial image sequence of the driver, then obtain a facial infrared image based on the facial image sequence, and use a 3D ToF sensor to obtain the three-dimensional coordinates of the driver's head. Then, we obtain a ToF point cloud based on the three-dimensional coordinates of the head. Then, we use the MobileNetV3 network to extract eye and mouth area features based on the facial infrared image, and use the 3D head posture solution to calculate the instantaneous values ​​and sliding window averages of the pitch and yaw angles to obtain state features.

[0030] For the extraction of biometric features, the steering wheel grip force distribution map can be obtained through the steering wheel grip force sensor, and then the grip force heat map is obtained based on the steering wheel grip force distribution map. The heart rate HR / respiratory rate RR signal is obtained through the millimeter wave bioradar, and then the original HR / RR signal (1 kHz sampling rate) is obtained based on the heart rate HR / respiratory rate RR signal. Then, after denoising through wavelet transform, HRV (Heart Rate Variability) and respiratory rhythm disorder index are obtained to obtain biometric features.

[0031] To extract irrational state features, a multi-directional microphone array is used to transcribe speech to text, which is then used to generate speech text. A pedal pressure sensor is used to obtain the timing sequence of pedal operations, and based on this timing sequence, a pedal operation curve is generated. Semantic deviation is then calculated (using a fine-tuned BERT (Bidirectional Encoder Representations from Transformers) model to compare the cosine similarity between the current speech and a preset command set). An LSTM (Long-Short Term Memory) network is then used to classify the operation pattern (the LSTM network identifies pedaling patterns, such as classifying "high-frequency braking" as abnormal behavior). Irrational state features are then derived.

[0032] The feature extraction process for state features, biometric features, and irrational state features also includes preprocessing of facial infrared images, ToF point clouds, grip force heat maps, raw HR / RR signals, voice text, and pedal operation curves, including time calibration and spatial calibration. Time calibration involves assigning a unified timestamp to each sensor data and compensating for transmission delays through Kalman filtering to ensure synchronization of irrational state data, physiological data, and state data on the timeline, thereby ensuring consistency across the multi-sensor data timeline. Spatial calibration involves calibrating the coordinate mapping relationship between the driver's head position and the radar monitoring area based on the driver's sitting posture.

[0033] Step S130, dynamically assigning weights of multiple driving strategy-related features; Dynamic weighting refers to a mechanism that adjusts the importance of feature parameters based on the real-time scenario, improving the adaptability of feature fusion. The real-time scenario can include both real-time environmental features and vehicle status features. Based on these real-time environmental features (such as weather and lighting) and real-time vehicle status features (such as speed and steering angle), a reliability prediction model (which can be trained using historical data) determines the weight of each feature. For example, in heavy rain, the weight of the abnormal steering wheel grip feature can be increased to 0.6, compared to only 0.3 in normal weather.

[0034] Step S140, performing weighted fusion on the multiple driving strategy associated features based on the weights of the multiple driving strategy associated features to obtain a driving strategy fusion feature; Weighted fusion involves linearly superimposing different features according to dynamically assigned weights. This can be achieved by element-by-element multiplication followed by addition. For example, the driver's heart rate variability feature and steering wheel grip feature can be added together with different weights. Feature extraction is performed on the weighted fusion feature matrix to obtain context-dependent driving strategy fusion features that reflect the influence of previous operations and predict subsequent trends. A bidirectional LSTM (Long Short-Term Memory) network can be used to extract contextual dependencies from a past period to a future period, such as from the past 10 seconds to the next 2 seconds.

[0035] Feature fusion can be achieved through a feature fusion network within a trained risk assessment model. This network fuses heterogeneous data from multiple sources to generate driving strategy fusion features. This network addresses the feature matching issues caused by heterogeneity in multimodal data and enhances robustness to environmental interference.

[0036] Step S150: determining a risk level assessment index based on the driving strategy fusion feature, and obtaining a risk assessment result based on the risk level assessment index and an index threshold; wherein the index threshold is generated based on the driver's personalized data and traffic environment data; The driving strategy fusion features are input into the prediction network of the trained risk assessment model. The prediction network determines the risk level assessment index by analyzing multiple driving strategy fusion features. The risk level assessment index is then compared with the index threshold to obtain the risk assessment result. The prediction network can adopt a Transformer network to predict the driver's future state trend, i.e., the risk assessment result. When the risk level assessment index is greater than the index threshold, it indicates that the trigger condition is met and the risk assessment result is generated. The index threshold refers to a dynamic standard generated based on the personalized data of different drivers and current traffic conditions. For example, a lower heart rate variability tolerance value is set for drivers with a history of heart disease.

[0037] The risk assessment results include risk level and confidence level. Different risk assessment results correspond to different risk levels. The risk level can be represented by L1, L2, L3, and L4. L1 represents normal state, L2 represents mild distraction / fatigue, L3 represents severe fatigue / sudden illness, and L4 represents loss of activity characteristics / no response.

[0038] S160: Generate an assisted driving strategy based on the risk assessment results.

[0039] An assisted driving strategy refers to a dynamic response plan generated based on risk assessment results. A preconfigured relationship between risk assessment results and assisted driving strategies can then be used to determine the appropriate assisted driving strategy based on the preconfigured relationship after the driver's risk assessment result is predicted using the driving risk assessment model.

[0040] The corresponding relationship between risk assessment results, trigger conditions, and assisted driving strategies is shown in Table 1.

[0041] Table 1

[0042] This application uses collaborative analysis of multi-dimensional data (driver status data, driver physiological data, and driver irrational status data) to reduce misjudgments caused by environmental factors and improve the robustness of state recognition. Furthermore, auxiliary strategies generated based on dynamic risk assessment results can automatically match the optimal intervention intensity in different scenarios. For example, in low-risk situations, only early warning prompts are provided, while in high-risk situations, active safety systems are directly triggered, achieving graded safety protection.

[0043] Through dynamic weight allocation, this application can adjust the degree of feature influence according to the real-time scenario, and by combining personalized data and traffic environment data to generate dynamic thresholds, it can capture the impact of driving scenario changes on feature importance in real time, making the risk assessment results more in line with the actual driving scenario, and improving the generation accuracy and response speed of auxiliary strategies.

[0044] In one embodiment, the indicator threshold is generated based on the driver's personalized data and traffic environment data, and the personalized data includes at least one of the following: the driver's physical health status data, the driver's driving habit data, and the driver's individual attribute data.

[0045] Among them, physical health status data refers to high-value risk data acquired through federated learning (such as whether the driver has a disease and changes in physiological characteristics after taking specific medications for the disease). Driving habit data refers to the operating patterns formed by drivers during long-term driving. This can be achieved by using the historical driving data acquisition module to record steering frequency, braking force, or number of lane changes, and is used to identify driving behavior characteristics. Driver individual attribute data refers to static information related to the driver's identity. Age, gender, or years of driving experience can be obtained through user registration information. Traffic environment data refers to real-time road condition information obtained through on-board cameras, radar, and V2X (vehicle-to-everything, V2X) communication equipment. It can be implemented using image recognition algorithms and vehicle networking protocols to capture dynamic parameters such as current road complexity, visibility, traffic density, and whether there is a traffic light.

[0046] For example, in rainy scenarios, if a driver's history of hypertension is detected, the heart rate variability threshold is adjusted from 15% below the personalized baseline to 13% below the personalized baseline. In highway scenarios, if driving habit data indicates a driver's preference for frequent lane changes, the trigger threshold for steering wheel grip entropy is raised to 1.2 times the standard value. Furthermore, visibility parameters from traffic environment data work synergistically with age data from individual attributes: for drivers over 60 years old driving in fog, the distraction index warning threshold is adjusted from the standard value of greater than 0.6 to greater than 0.5. By establishing a multidimensional mapping relationship between personalized data and traffic environment data, composite indicator thresholds can be generated to accommodate a wider range of scenarios.

[0047] In one embodiment, the risk assessment result includes a risk assessment level and a confidence level corresponding to the risk assessment level; the assisted driving method also includes: performing a first verification of the risk assessment result based on the confidence level and the confidence level threshold; if the first verification passes, performing a second verification of the risk assessment result based on a preset redundant rule.

[0048] Confidence refers to the quantitative value of the credibility of the model's output risk assessment level, reflecting data quality and the stability of the model's predictions. Confidence thresholds refer to the preset screening criteria. Redundancy rules refer to the preset auxiliary verification logic conditions used to re-verify results that have passed preliminary verification.

[0049] When the driving risk assessment model outputs a result including the risk level and confidence level, the confidence level is first compared with the preset threshold. For example, if the confidence threshold is set to 0.9, when the confidence level of the model output is greater than 0.9, the risk assessment level output by the driving risk assessment model is considered correct. For the results that pass the threshold verification, the redundant rules are further called for secondary verification. For example, when it is determined that the driver is in severe fatigue / sudden illness, it is necessary to simultaneously detect whether the sudden drop in the steering wheel grip entropy value of the vehicle steering wheel exceeds the preset threshold, or whether the pupil dilation lasts for more than 5 seconds and whether the breathing rate is abnormal (less than 8 times / minute or greater than 30 times / minute). If both verifications pass, it is considered that the driver is indeed in severe fatigue / sudden illness, and the corresponding assisted driving strategy is generated at this time.

[0050] Of course, for vehicles involving intelligent driving, in addition to the driving risk assessment model, other risk models are also set up. Other risk models can be models that update weights based on indicators such as driver takeover delay time and braking distance deviation.

[0051] The embodiments of the present application solve the problem of insufficient credibility of risk assessment results due to environmental interference, reduce the misjudgment rate through a double verification mechanism, and avoid premature intervention or delayed response of the assisted driving system due to incorrect risk assessment.

[0052] In one embodiment, the risk level assessment indicator includes at least one of the following: attention distraction index, heart rate variability, pupil dilation duration, respiratory rate, steering wheel grip entropy, and cardiac arrest time.

[0053] Among them, the attention distraction index refers to the pupil diameter, blink interval, and head posture Euler angle calculation collected by infrared cameras and 3D ToF sensors, which is used to quantify the degree of driver's attenuation of the road environment. Heart rate variability is used to reflect the driver's fatigue and stress response level. It is obtained by detecting heart rate HR, respiratory rate RR, and body movement amplitude (BAM) signals through millimeter wave bioradar. Pupil dilation duration refers to the rate of change of pupil diameter data. Respiratory rate refers to chest movement cycle data, which is used to monitor abnormal breathing patterns caused by tension or illness. Steering wheel grip entropy refers to the steering wheel grip pressure distribution data collected by the pressure sensor array, which is used to characterize the driver's stability in vehicle control. Cardiac arrest time is used to warn of sudden cardiovascular disease risks by identifying abnormal events in which the RR interval exceeds the set threshold.

[0054] In one embodiment, the multiple driving strategy association factors include a first driving strategy association factor stored locally and a second driving strategy association factor stored in the cloud. Obtaining the multiple driving strategy association factors includes: fusing the first driving strategy association factor and the second driving strategy association factor based on a federated learning method to obtain the multiple driving strategy association factors.

[0055] Federated learning, a distributed machine learning framework, can be implemented using an encrypted parameter aggregation mechanism. Only model parameters are exchanged between the local device and the cloud, without transmitting raw data. The first driving strategy-related factor stored locally includes the driver's physiological data. The second driving strategy-related factor stored in the cloud includes state data and irrational state data. The feature fusion process is accomplished through a horizontal federated learning architecture, enabling joint modeling of the feature space while maintaining physical isolation of the data. (Gaussian noise (ε = 0.1) is added before the features are uploaded to the cloud to ensure that individual data cannot be traced.) Among them, all feature data are standardized through federated learning and uploaded to the cloud server through the differential privacy mechanism. The desensitized feature data is uploaded to the cloud after a set time period to participate in the training of the global driving risk assessment model.

[0056] The local on-board terminal collects the driver's facial expression and heart rate data in real time to form the first driving strategy correlation factor, which is stored in the vehicle's local encrypted memory. The cloud receives traffic flow data and weather data to form the second driving strategy correlation factor. A local-cloud communication link is established through the federated learning coordinator, and the gradient parameters of the local model training are encrypted and transmitted with the cloud model parameters. After parameter aggregation is completed, the updated global model is distributed to the local and cloud nodes for iterative optimization.

[0057] In one embodiment, a safety coprocessor concurrently receives raw driver status data, physiological data (such as bioradar HR signals), and irrational driver status data. Upon detecting a threshold (e.g., "HR = 0 for 5 consecutive seconds"), it directly triggers the MRM (Minimum Risk Maneuver), bypassing potential main system failures. By implementing a safety coprocessor (ASIL-D) independent of the main control system, the Minimum Risk Maneuver (MRM) can be executed even in the event of a main system failure if the loss of vital signs (e.g., cardiac arrest) is detected, ensuring driving safety. The ASIL-D safety coprocessor can directly take over, disconnecting control of the main system, triggering emergency braking (ERB, deceleration 6m / s²), and calling for assistance (eCall + GPS coordinates).

[0058] In one embodiment, a power supply redundancy function is also provided, where the safety coprocessor and the bio-radar are powered by an independent backup power supply (supercapacitor), ensuring that they can still operate for at least 60 seconds when the main battery fails.

[0059] In one embodiment, the weights of multiple driving strategy-related features are dynamically assigned, including: Step S210, acquiring real-time environmental characteristics and real-time vehicle status characteristics; Real-time environmental characteristics include light intensity (lux), rainfall level (mm / h), fog concentration (visibility m), temperature (°C), wind speed (m / s), etc. Real-time vehicle status characteristics refer to the real-time parameters of the vehicle during operation. Data such as vehicle speed (km / h), window status (open / closed), air conditioning mode, and vehicle body vibration intensity can be read through the vehicle controller area network bus to represent the vehicle's current state.

[0060] Step S220, determining a reliability score of the driving strategy-related feature based on the real-time environment feature and the real-time vehicle state feature; Reliability scores can be derived from a reliability prediction model, a weighted prediction model trained using a machine learning algorithm. This model can employ a long short-term memory network or a convolutional neural network to jointly model historical driving data and real-time data. This model is used to predict the reliability scores of different driving strategy-related features in the current scenario. The reliability score represents the reliability of the driving strategy-related features. A higher reliability score indicates a higher reliability of the driving strategy-related features, while a lower reliability score indicates a lower reliability of the driving strategy-related features.

[0061] Step S230 , determining the weight of each driving strategy-related feature based on the reliability score to obtain a dynamic weight matrix; A high reliability score indicates that the associated feature has high reliability. The corresponding weight is determined based on the reliability score of each driving strategy associated feature. A high reliability score means a high weight, and a low reliability score means a low weight, so that a dynamic weight matrix can be determined.

[0062] The dynamic weight matrix refers to the multi-dimensional weight distribution data output by the reliability prediction model. The reliability coefficients of each feature can be normalized into weight values ​​through matrix operations, which can be used to achieve adaptive adjustment of feature weights in different driving scenarios.

[0063] Step S240 : Dynamically assigning weights of multiple driving strategy-related features based on a dynamic weight matrix.

[0064] The weight settings of the same sensor (different associated features) in different scenarios are shown in Table 2.

[0065] Table 2

[0066] During vehicle driving, real-time environmental and vehicle status features are continuously collected and input into a pre-trained reliability prediction model. This reliability prediction model calculates the reliability score of each driving strategy-related feature (such as heart rate variability and steering wheel grip entropy) based on the current road environment and vehicle motion trends. The reliability score is normalized to generate a dynamic weight matrix, in which each element corresponds to the real-time weight value of a driving strategy-related feature. For example, in a rainstorm scenario, the model may reduce the weight of visual features (such as pupil dilation duration) while increasing the weight of tactile features (such as steering wheel grip entropy).

[0067] This application uses dual perception of the real-time environment and vehicle status, combined with the dynamic prediction capabilities of machine learning models, to automatically adjust weight distribution based on sudden events such as sudden drops in visibility and sudden braking. For example, if a vehicle suddenly deviates from its lane, the weight of the steering wheel operation feature is immediately increased, allowing for faster identification of the driver's status during the risk assessment phase.

[0068] In one embodiment, the driving strategy fusion features include short-term features, mid-term features, and long-term features.

[0069] Among them, short-term features can be driver status data collected in real time, which can be realized by the frequency of changes in steering wheel grip force and the amplitude of instantaneous heart rate fluctuations, and are used to reflect the driver's immediate reaction ability within a time window of seconds. Medium-term features refer to driving behavior trends over a period of time in the past, which can be realized by the average breathing frequency and the rate of change of the attention distraction index in the past five minutes, and are used to capture the driver's state fluctuations within a time window of minutes. Long-term features refer to the stable behavior patterns accumulated by the driver during multiple driving processes, which can be realized by the mean steering wheel grip force entropy and the distribution law of cardiac arrest time in historical driving records, and are used to characterize the driver's individual driving habits within a time window of hours or days.

[0070] When generating driving strategy fusion features, features of different time spans are layered. For example, the duration of pupil dilation collected in real time is used as a short-term feature, the standard deviation of steering wheel angles over the past three minutes is used as a medium-term feature, and the baseline of heart rate variability over the past week is used as a long-term feature. Subsequently, a weighted fusion method is used in the time dimension, such as a sliding window mechanism, to assign dynamic weights to features of different time spans. This ensures that the fused features contain both immediate risk signals of the current driving environment and the evolving trends of the driver's physiological state, while also taking into account long-term driving behavior patterns. This allows for a comprehensive assessment of sudden risks, gradual risks, and potential habitual risks when generating risk level assessment indicators.

[0071] This application can effectively distinguish the risk differences caused by instantaneous operational errors and long-term behavioral pattern defects. In continuous driving scenarios on highways, it can accurately identify the lane departure risks caused by short-term distraction, while combining medium-term characteristics to determine whether there is a trend of fatigue driving, and then use long-term characteristics to evaluate whether the driver has habits.

[0072] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0073] Figure 3 FIG. 1 is a block diagram of a driving assistance device according to an embodiment of the present application. Figure 3 As shown, a driving assistance device includes: a data acquisition module 310 for acquiring a plurality of driving strategy-related factors, wherein the plurality of driving strategy-related factors include at least two of the driver's state data, the driver's physiological data, and the driver's irrational state data; A feature extraction module 320 is used to extract features from multiple driving strategy related factors to obtain multiple driving strategy related features; A weight allocation module 330 is used to dynamically allocate weights of multiple driving strategy-related features; a feature fusion module 340 for performing weighted fusion on the multiple driving strategy-related features based on the weights of the multiple driving strategy-related features to obtain a driving strategy fusion feature; a risk assessment module 350 for determining a risk level assessment index based on the driving strategy fusion feature, and obtaining a risk assessment result based on the risk level assessment index and an index threshold; wherein the index threshold is generated based on the driver's personalized data and traffic environment data; The strategy generation module 360 ​​is used to generate an assisted driving strategy based on the risk assessment results.

[0074] It should be noted that the assisted driving device provided in the above embodiments and the assisted driving method provided in the above embodiments are based on the same concept. The specific manner in which each module and unit performs operations has been described in detail in the method embodiments and will not be repeated here. In actual applications, the assisted driving device provided in the above embodiments can, as needed, allocate the above functions to different functional modules, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0075] An embodiment of the present application also provides a device comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by one or more processors, the memory implements the assisted driving method in the above embodiment.

[0076] Embodiments of the present application also provide one or more machine-readable media having instructions stored thereon, which, when executed by one or more processors, enable the processors to execute the assisted driving method in the above-mentioned embodiment.

[0077] Figure 4 FIG1 shows a schematic diagram of a computer system suitable for implementing a memory according to an embodiment of the present application. It should be noted that: Figure 4 The computer system of the memory shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0078] like Figure 4As shown, computer system 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on programs stored in read-only memory (ROM) 402 or programs loaded from storage into random access memory (RAM) 403. RAM also stores various programs and data required for system operation. CPU 401, ROM 402, and RAM 403 are interconnected via bus 404. An input / output (I / O) interface 405 is also connected to bus 404.

[0079] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, mouse, and the like; an output section 407 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and speakers; a storage section 408 including devices such as a hard disk; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. Removable media 411, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 410 as needed, so that computer programs read from the media can be installed in the storage section 408 as needed.

[0080] In particular, according to embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for executing the aforementioned assisted driving method. In such embodiments, the computer program can be downloaded and installed from a network via a communication component and / or installed from removable media 411. When executed by the central processing unit (CPU) 401, the computer program performs the various functions defined in the system of the present application.

[0081] It should be noted that the computer-readable medium described in the embodiments of this application may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may, for example, be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM) 403, read-only memory (ROM) 402, erasable programmable read-only memory (EPROM), flash memory, optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. This propagated data signal may take a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. A computer program embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0082] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. Among them, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0083] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. In some cases, the names of these units do not constitute limitations on the units themselves.

[0084] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon. When executed by a computer processor, the computer program causes the computer to perform the aforementioned assisted driving method. The computer-readable storage medium may be included in the memory described in the above embodiments, or may exist independently and not be incorporated into the memory.

[0085] Another aspect of the present application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the assisted driving method provided in each of the above embodiments.

[0086] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, any equivalent modifications or alterations accomplished by a person of ordinary skill in the art without departing from the spirit and technical concepts disclosed in this application shall be covered by the claims of this application.

Claims

1. A driving assistance method, characterized in that: The assisted driving method includes: Acquiring a plurality of driving strategy-related factors, the plurality of driving strategy-related factors comprising at least two of the driver's state data, the driver's physiological data, and the driver's irrational state data; Extract features of multiple driving strategy related factors to obtain multiple driving strategy related features; Dynamically assign weights to multiple driving strategy-related features; performing weighted fusion on the multiple driving strategy associated features based on the weights of the multiple driving strategy associated features to obtain a driving strategy fusion feature; Determining a risk level assessment index based on the driving strategy fusion feature, and obtaining a risk assessment result based on the risk level assessment index and an index threshold; wherein the index threshold is generated based on the driver's personalized data and traffic environment data; An assisted driving strategy is generated based on the risk assessment results.

2. The assisted driving method according to claim 1, characterized in that: The personalized data includes at least one of the following: driver's physical health status data, driver's driving habit data, and driver's individual attribute data.

3. The assisted driving method according to claim 1, characterized in that: The risk assessment result includes a risk assessment level and a confidence level corresponding to the risk assessment level; the assisted driving method further includes: Performing a first verification on the risk assessment result based on the confidence level and the confidence threshold; When the first verification is passed, the risk assessment result is verified for the second time based on a preset redundancy rule.

4. The assisted driving method according to claim 1, characterized in that: The risk level assessment indicator includes at least one of the following: Attention distraction index, heart rate variability, pupil dilation duration, respiratory rate, steering wheel grip entropy, and cardiac arrest time.

5. The assisted driving method according to claim 1, characterized in that: The multiple driving strategy association factors include a first driving strategy association factor stored locally and a second driving strategy association factor stored in the cloud. Obtaining the multiple driving strategy association factors includes: fusing the first driving strategy association factor and the second driving strategy association factor based on a federated learning method to obtain the multiple driving strategy association factors.

6. The assisted driving method according to claim 1, characterized in that: The dynamically allocating weights of the plurality of driving strategy-related features includes: Obtain real-time environmental features and real-time vehicle status features; Determine the reliability score of driving strategy-related features based on real-time environmental features and real-time vehicle status features; Based on the reliability score, the weight of each driving strategy-related feature is determined to obtain a dynamic weight matrix; The weights of multiple driving strategy-related features are dynamically allocated based on a dynamic weight matrix.

7. The assisted driving method according to claim 1, characterized in that: The driving strategy fusion features include short-term features, mid-term features and long-term features.

8. A driving assistance device, characterized in that: The auxiliary driving device includes: a data acquisition module, configured to acquire a plurality of driving strategy-related factors, wherein the plurality of driving strategy-related factors include at least two of the driver's state data, the driver's physiological data, and the driver's irrational state data; A feature extraction module is used to extract features of multiple driving strategy related factors to obtain multiple driving strategy related features; A weight allocation module is used to dynamically allocate weights of multiple driving strategy-related features; a feature fusion module, configured to perform weighted fusion on the plurality of driving strategy-related features based on the weights of the plurality of driving strategy-related features to obtain a driving strategy fusion feature; a risk assessment module, configured to determine a risk level assessment index based on the driving strategy fusion feature, and obtain a risk assessment result based on the risk level assessment index and an index threshold; wherein the index threshold is generated based on the driver's personalized data and traffic environment data; A strategy generation module is used to generate an assisted driving strategy based on the risk assessment results.

9. A driving assistance device, characterized in that: include: one or more processors; and A memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the memory implements the method according to any one of claims 1 to 7.

10. A machine-readable medium, characterized in that Instructions are stored thereon, which, when executed by one or more processors, cause the processors to perform the method according to any one of claims 1 to 7.

Images