Driving intention grading intervention method, system and equipment based on multiple factors and medium

Abstract

The invention provides a multi-factor-based driving intention grading intervention method, system and device and a medium, and belongs to the technical field of intelligent auxiliary driving, and the method comprises the steps: obtaining a first type of factor data in a current driving scene, inputting the first type of factor data into a preset risk early warning model, and outputting a risk early warning index; acquiring a second type of factor data in the current driving scene, inputting the first type of factor data and the second type of factor data into a preset intention recognition model together, and outputting probability distribution and a consistency score of a driving intention; when the consistency score is lower than a preset score threshold, correcting probability distribution of the driving intention; and based on the risk early warning index and the corrected probability distribution of the driving intention, executing intervention strategies of different levels. Through the technical scheme in the embodiment of the invention, the driving intention can be analyzed in combination with multiple factors, and graded intervention can be performed in time.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent auxiliary driving, and in particular to a driving intention hierarchical intervention method and system based on multiple factors, a device and a medium. BACKGROUND

[0002] During driving, the driver is easily disturbed by various factors, resulting in the execution of incorrect driving actions. In particular, in complex driving scenarios, such as the sudden appearance of an obstacle in front of the vehicle, the driver should have performed the action of stepping on the brake, but instead performed the action of stepping on the accelerator, increasing the risk of accidents.

[0003] Currently, some vehicles have been equipped with an auxiliary driving system that not only can automatically cruise, but also can obtain data information during driving through sensors deployed around the vehicle, and even automatically perform actions such as emergency braking in emergency situations to avoid accidents. However, the existing auxiliary driving system cannot accurately understand the driving intention of the driver, and thus incorrectly executes intervention strategies.

[0004] Therefore, there is an urgent need to design a scheme that can accurately analyze the driving intention of the driver and then execute corresponding intervention strategies. SUMMARY

[0005] The present application aims to solve at least one of the technical problems existing in the prior art, and proposes a driving intention hierarchical intervention method, system, device and storage medium based on multiple factors.

[0006] In a first aspect, the present application embodiment provides a driving intention hierarchical intervention method based on multiple factors, comprising:

[0007] obtaining first factor data under the current driving scenario, and inputting the first factor data into a pre-set risk warning model to output a risk warning index;

[0008] obtaining second factor data under the current driving scenario, and inputting the first factor data and the second factor data into a pre-set intention recognition model to output a probability distribution of the driving intention and a consistency score;

[0009] when the consistency score is lower than a pre-set score threshold, correcting the probability distribution of the driving intention until the consistency score is greater than or equal to the pre-set score threshold; and based on the risk warning index and the corrected probability distribution of the driving intention, executing intervention strategies of different levels.

[0010] In some embodiments, the first factor data includes environmental data around the vehicle, health status data of the driver and weather data;

[0011] The second type of factor data includes operation data of the driver.

[0012] In some embodiments, the risk warning model is pre-configured with a first weighting strategy, which is used to assign a fixed weight to each type of factor data in the first type of factor data.

[0013] The risk warning model obtains a risk warning index by weighting and summing the weights of various factor data.

[0014] In some embodiments, the intention recognition model is pre-configured with a second weighting strategy, which is used to dynamically assign weights to the first type of factor data and the second type of factor data for each driving intention.

[0015] The intention recognition model obtains a probability of each driving intention by weighting and summing the weights of various factor data.

[0016] In some embodiments, the intention recognition model is also pre-configured with a score mapping table, in which different risk warning indexes and different probabilities of driving intentions are pre-stored.

[0017] The driving intention with the highest corresponding probability in the probability distribution of the driving intention is taken as a first driving intention.

[0018] Based on the risk warning index output by the risk warning model, the first driving intention, and the probability of the first driving intention, the consistency score is obtained by searching the score mapping table.

[0019] In some embodiments, the types of driving intentions include at least: smooth driving, acceleration overtaking, deceleration avoidance, emergency braking, and abnormal operation.

[0020] In some embodiments, based on the risk warning index and the probability distribution of the corrected driving intention, different levels of intervention strategies are executed, including:

[0021] The driving intention with the highest corresponding probability in the probability distribution of the driving intention is taken as a second driving intention.

[0022] When the risk warning index is less than a preset risk threshold, the engine or transmission of the vehicle is controlled according to the second driving intention.

[0023] When the risk warning index is greater than or equal to the preset risk threshold, the brake and power system of the vehicle are controlled according to the second driving intention.

[0024] In a second aspect, embodiments of the present application provide a multi-factor-based driving intention hierarchical intervention system, which comprises:

[0025] a first output unit, configured to acquire first type factor data in a current driving scene, and input the first type factor data into a pre-set risk warning model to output a risk warning index;

[0026] a second output unit, configured to acquire second type factor data in the current driving scene, and input the first type factor data and the second type factor data into a pre-set intention recognition model to output a probability distribution of a driving intention and a consistency score;

[0027] an intention correction unit, configured to correct the probability distribution of the driving intention until the consistency score is greater than or equal to a pre-set score threshold when the consistency score is lower than the pre-set score threshold, and execute intervention strategies of different levels based on the risk warning index and the corrected probability distribution of the driving intention.

[0028] In a third aspect, an embodiment of the present application provides an electronic device, and the electronic device comprises:

[0029] at least one processor; and a memory connected with the at least one processor in communication;

[0030] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the steps of the method of any embodiment of the present application.

[0031] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute the steps of the method of any embodiment of the present application when executed by the processor.

[0032] Compared with the prior art, the present application has the following beneficial effects:

[0033] The application provides a multi-factor-based driving intention hierarchical intervention method, which comprises the following steps: firstly, acquiring first factor data in a current driving scene, inputting the first factor data into a pre-set risk warning model, and outputting a risk warning index; then, acquiring second factor data in the current driving scene, inputting the first factor data and the second factor data into a pre-set intention recognition model, and outputting a consistency score between the first factor data and the second factor data and a probability distribution of driving intention; finally, when the consistency score is lower than a pre-set score threshold, the probability distribution of driving intention is corrected until the consistency score is greater than or equal to the pre-set score threshold; and based on the risk warning index and the corrected probability distribution of driving intention, different levels of intervention strategies are executed. Through the technical scheme of the application, various factor data such as environmental data, state data, weather data and operation data are fused, and compared with single-factor decision, the risk can be more accurately judged, especially in complex weather and road conditions, the road surface wetness, traffic density and driver emotion are comprehensively considered, and deceleration or avoidance is implemented in advance, so that the accident risk is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiments description. Obviously, the drawings in the following description are only preferred embodiments of the application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0035] Figure 1 A flowchart of a multi-factor-based driving intention hierarchical intervention method provided by the embodiment of the application;

[0036] Figure 2 A flowchart of a consistency scoring method based on multi-factor data provided by the embodiment of the application;

[0037] Figure 3 A flowchart of a hierarchical intervention method based on corrected driving intention provided by the embodiment of the application;

[0038] Figure 4 A structure block diagram of a multi-factor-based driving intention hierarchical intervention system provided by the embodiment of the application;

[0039] Figure 5 A structure block diagram of an electronic device provided by the embodiment of the application. DETAILED DESCRIPTION

[0040] With reference to the drawings and the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of the present application.

[0041] In order for those skilled in the art to better understand the technical solutions of the present application, the exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to help understanding. They should be considered only as exemplary. Therefore, those of ordinary skill in the art should realize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Also, for the sake of clarity and conciseness, the description below omits the description of well-known functions and structures.

[0042] In the case of no conflict, each embodiment of the present application and each feature in the embodiments can be combined with each other.

[0043] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0044] The terms used herein are only used to describe specific embodiments, and are not intended to limit the present application. As used herein, the singular forms "a" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the terms "comprise" and / or "consist of," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. The terms "connected" or "coupled" and / or similar terms are not limited to a physical or mechanical connection, but can include an electrical connection, whether direct or indirect.

[0045] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present application, and will not be interpreted in an overly literal or overly formal sense unless expressly so defined herein.

[0046] The collection, storage, use, processing, transmission, provision and disclosure of the user personal information in the technical solution of the present application comply with relevant laws and regulations and do not violate public order and good customs. The use of user data in the technical solution complies with relevant national laws and regulations (for example, "Information Security Technology Personal Information Security Specification" and the like). For example, appropriate measures are taken for personal information access control; the display of personal information is limited according to regulations; the use purpose of personal information does not exceed the direct or reasonably related range; and the use of personal information eliminates explicit identity orientation and avoids precise positioning to a specific individual.

[0047] Before introducing the technical solution of the present application, it needs to be explained that there are the following several schemes for intervening in driving in the related art:

[0048] 1) Scheme one: equipped with an advanced driver assistance system, using millimeter wave or laser radar to monitor the distance and speed of the preceding vehicle, automatically adjusting the vehicle speed to maintain a safe distance; through a camera or radar to perceive the front obstacles, when predicting a collision and the driver does not brake, automatically applying the brake. In addition, some vehicles are equipped with a driver fatigue monitoring system, which captures facial features (such as eye closure degree, blink frequency) through an in-vehicle camera, or monitors the change in steering wheel operation frequency to judge the driver's fatigue state and issue a warning. However, this scheme only makes decisions based on a few factors such as the distance, speed of the preceding vehicle or obstacles, and the driver's state monitoring is insufficient, which may result in missing the best intervention opportunity.

[0049] 2) Scheme two: equipped with a vehicle dynamic control system, which monitors the vehicle's yaw rate and lateral acceleration through sensors, and when it detects understeering or oversteering, automatically brakes the wheels and adjusts the engine torque to stabilize driving. In addition, some intelligent traffic systems use vehicle networking technology to obtain road congestion, signal light information, and slow down in advance on congested road sections. However, this scheme only adjusts the vehicle's own dynamics and does not consider comprehensive analysis under multiple factors.

[0050] In summary, in the process of intervening in driving intention in the existing scheme, since each factor is independent, the intervention strategy is single and not accurate due to insufficient integration.

[0051] To solve at least one of the technical problems existing in the related art, the present application provides a multi-factor-based driving intention hierarchical intervention method. Figure 1 A flowchart of a multi-factor-based driving intention hierarchical intervention method provided by an embodiment of the present application is shown in the figure. The method is applicable to the intervention of driving intention in complex weather, congestion and other automobile driving scenarios. The method can be executed by a multi-factor-based driving intention hierarchical intervention system, which can be implemented in software and / or hardware and can be configured in an electronic device.

[0052] AsFigure 1 The method specifically comprises the following steps:

[0053] S1, acquiring first type factor data under a current driving scene, and inputting the first type factor data into a pre-set risk warning model to output a risk warning index.

[0054] The first type factor data comprises environmental data around the vehicle, health state data of the driver, and weather data.

[0055] The environmental data around the vehicle at least comprises position information and speed information of surrounding vehicles or obstacles, and road condition information. Exemplarily, a millimeter wave radar, a laser radar, and a camera can be deployed on the vehicle body to comprehensively perceive the surrounding conditions of the vehicle. The millimeter wave radar and the laser radar can be installed at the rear of the front bumper or the roof of the vehicle, etc., to ensure that the detection field of view is not blocked, and further can adopt shock absorption and waterproof and dustproof design to ensure stable work in complex road conditions and bad weather. The camera can consider the range of the visual angle and the quality of image acquisition to avoid blind spots and image distortion.

[0056] Specifically, the millimeter wave radar is used to detect long-distance targets and complete target speed measurement, the laser radar is used to accurately acquire three-dimensional coordinates of target objects, and the camera and image recognition technology are used to identify lane lines, traffic signs, and various road conditions. In some application scenarios, an ultrasonic radar can also be used as an auxiliary, especially when detecting obstacles at a close distance, to perceive the relative position of the obstacles. After obtaining the relative position matrix of the surrounding vehicles and obstacles by using the radar technology, the noise in the relative position matrix can be smoothed by using Kalman filtering to ensure the stability of the data.

[0057] The health state data of the driver at least comprises blink frequency, pupil diameter change rate, facial muscle movement amplitude, voice, heart rate, pulse, blood pressure, and blood oxygen of the driver. Exemplarily, a plurality of sensors can be arranged in the vehicle to comprehensively judge the health and mental state of the driver in combination with multi-source data. The camera can be used to monitor the facial expression and eye state to judge the fatigue state, the bioelectric sensor placed in the steering wheel can be used to collect the heart rate and pulse, and the microphone can be used to analyze the voice features to evaluate the emotion. For the driver health monitoring sensor, such as the bioelectric sensor on the steering wheel, it is required to ensure good contact with the hands of the driver and not to affect the normal operation of the driver, and special materials and processes are adopted to ensure the sensitivity and durability of the sensor.

[0058] Specifically, the camera can be used to extract the blink frequency, the pupil diameter change rate and the facial muscle movement amplitude to determine the fatigue state of the driver, the bioelectricity sensor can be used to analyze the HRV (e.g., the NN50 index) to determine the health state of the driver, and the voice data can be used to extract the MFCC to determine the fatigue, tension and emotional state. In some application scenarios, the wearable device (e.g., a smart bracelet, a smart helmet, etc.) can be combined with the in-vehicle sensor to monitor the health state of the driver. The wearable device can accurately obtain the physiological data of the driver, such as the blood pressure and the blood oxygen, to make up for the deficiency of the in-vehicle sensor.

[0059] The weather data at least includes the rainfall, the illumination, the humidity, the air temperature, the air pressure and the wind speed. Specifically, the meteorological sensor can be deployed on the vehicle body to collect the real-time weather information, and the vehicle networking can be used to obtain the more comprehensive data such as the air temperature, the air pressure, the wind speed and the weather forecast from the external meteorological server. Further, the rainfall data and the illumination data can be converted into the precipitation level and the visibility index, and combined with the road friction coefficient of the V2X to form a three-dimensional weather influence factor vector.

[0060] One of the cores of the embodiment is that the millimeter wave radar, the laser radar, the camera, the bioelectricity sensor, the meteorological sensor and other types of sensors are used to collect the data of the surrounding environment of the vehicle, the health state of the driver and the weather condition, to realize the all-around information acquisition.

[0061] Further, the first weighting strategy is pre-configured in the risk warning model, and the first weighting strategy is used to assign a fixed weight to each factor data in the first type of factor data. The risk warning model obtains the risk warning index by weighted summation of the weights of various factor data.

[0062] Specifically, the environmental danger degree, the driver state risk degree and the weather influence degree are configured for the environment data, the state data and the weather data in the first type of factor data, respectively. The environmental danger degree can be obtained by weighting the distance to the preceding vehicle, the relative speed, the obstacle type and the lane line offset; the driver state risk degree can be obtained by evaluating the physiological and behavioral characteristics of the driver; and the weather influence degree can be quantified according to the influence of the weather condition on the road friction and the visibility. The fixed weights assigned to the environment data, the state data and the weather data are combined with the values corresponding to the respective indexes of the environmental danger degree, the driver state risk degree and the weather influence degree, and the weighted summation is performed to obtain the risk warning index.

[0063] It should be noted that the risk warning index reflects the potential risk situation dominated by the environment data, the state data and the weather data. For example, the risk of vehicle skidding in heavy rain or snow weather, the risk of vehicle out of control in the driver fatigue state, etc.

[0064] S2, acquire second type factor data in the current driving scene, and input the first type factor data and the second type factor data into a pre-set intention recognition model to output a probability distribution of driving intention and a consistency score.

[0065] The second type factor data includes operation data of the driver.

[0066] The operation data of the driver at least includes pedal information. For example, the pedal depth, the change rate, the pressure peak value and the duration are extracted by the displacement and pressure sensor. For example, the sudden acceleration corresponds to "the depth is greater than 80% and the change rate is greater than 50% / s", and the smooth acceleration corresponds to "the depth is 30-50% and the change rate is less than 20% / s".

[0067] Specifically, a high-precision displacement and pressure sensor can be installed on the pedal to collect the stepping depth, the speed and the pressure in real time, and accurately understand the acceleration and braking intention of the driver.

[0068] Further, a second weighting strategy is pre-configured in the intention recognition model, and the second weighting strategy is used to dynamically allocate the weight of the first type factor data and the second type factor data for each driving intention. The intention recognition model obtains the probability of each driving intention by weighting and summing the weights of various factor data.

[0069] In some embodiments, a deep neural network, a decision tree algorithm or a support vector machine algorithm and other machine learning technologies can be used to construct a driving intention prediction model, and the model is trained and optimized through a large amount of actual driving data, so that it can accurately predict the intention of the driver according to the characteristic parameters and evaluate the risk level of the current driving scene.

[0070] In a specific application scenario, the intention recognition model is configured to adopt a hybrid model of deep belief network + attention mechanism.

[0071] The network structure of the intention recognition model is as follows:

[0072] Input layer: multi-dimensional fusion feature vector (including environmental data, state data, weather data and operation data).

[0073] Hidden layer: 3 layers of restricted Boltzmann machine (RBM) are set, and the number of nodes of each layer is 64, 32 and 16 respectively. The data distribution features are learned by unsupervised pre-training (for example, the strong correlation between rainy day and slow pedal operation).

[0074] Attention layer: assign weights to the multi-dimensional features of the hidden layer output, dynamically strengthen the influence of key features. For example, the weight of the "driver's operation data" is 0.3 when identifying the "acceleration overtaking intention", and the weight of the "environmental data around the vehicle" is 0.4 when identifying the "emergency braking intention".

[0075] Output layer: use a softmax classifier to output the probability distribution of driving intention.

[0076] The model training strategy of the intention recognition model is as follows:

[0077] Use the "pre-training + fine-tuning" mode: first pre-train the RBM layer with unlabeled data (50,000 samples), and then fine-tune the whole network with labeled data (50,000 samples, manually labeled intention categories) through back propagation to optimize the cross-entropy loss function.

[0078] Introduce adversarial sample training: add a small perturbation (such as ±5%) to the pedal depth and other features to improve the robustness of the model to sensor noise and operation errors.

[0079] In some embodiments, Kalman filtering algorithm, Bayesian network algorithm or D-S evidence theory algorithm can be used to fuse and process multi-source heterogeneous data collected by different sensors, eliminate noise and redundancy, and integrate a unified effective data set to provide accurate basis for subsequent analysis.

[0080] In some embodiments, the intention recognition model is also pre-configured with a score mapping table, which pre-stores the scores between different risk warning indexes and different probability driving intentions.

[0081] Figure 2 The flowchart of the method for consistency scoring based on multi-factor data provided by the embodiment of the application is shown in Figure 2 As shown in the figure, the method for consistency scoring includes:

[0082] S201, the driving intention with the highest corresponding probability in the probability distribution of the driving intention is taken as the first driving intention;

[0083] S202, based on the risk warning index output by the risk warning model, the first driving intention and the probability of the first driving intention, the score mapping table is searched to obtain the consistency score.

[0084] It should be noted that the consistency score reflects the logical consistency between the risk warning index output by the risk warning model and the probability distribution of the driving intention output by the intention recognition model. For example, in rainy weather data, the driver accelerates driving, and the risk warning index output by the risk warning model is high, and the probability of acceleration overtaking is the highest in the probability distribution of the driving intention output by the intention recognition model. The consistency score in this case is set to a lower value.

[0085] Further, when the consistency score is lower than the preset score threshold, the weights of the first type factor data and the second type factor data in the intention recognition model are adjusted to correct the probability distribution of the driving intention until the consistency score is greater than or equal to the preset score threshold.

[0086] In some embodiments, the types of driving intention at least include: smooth driving, acceleration overtaking, deceleration avoiding, emergency braking, and abnormal operation.

[0087] S3, when the consistency score is lower than the preset score threshold, the probability distribution of the driving intention is corrected until the consistency score is greater than or equal to the preset score threshold; and based on the risk warning index and the corrected probability distribution of the driving intention, different levels of intervention strategies are executed.

[0088] Specifically, when the consistency score is lower than the preset score threshold, the weights of the first type factor data and the second type factor data in the intention recognition model are adjusted, and the probability distribution of the driving intention and the consistency score are re-output.

[0089] For example, after the intention recognition model outputs the probability distribution of the driving intention, the consistency score is modified again. For example, when the model identifies that the probability of "acceleration intention" is 0.7, but the "risk warning index" is greater than 0.8 (such as the distance to the front vehicle is less than 50m and the weather is heavy rain), at this time the consistency score obviously deviates, triggering intention correction, and the probability of the corrected acceleration overtaking intention is down-regulated to 0.3, and the probability of the deceleration avoiding intention is up-regulated to 0.6. Finally, the "risk-intention" binary decision result (such as low risk-smooth driving, high risk-emergency braking demand) is output, which provides a direct basis for hierarchical intervention.

[0090] Figure 3 A flowchart of a hierarchical intervention method based on corrected driving intention provided by an embodiment of the present application is shown in FIG. 1. Figure 3 As shown in FIG. 1, based on the risk warning index and the corrected probability distribution of the driving intention, different levels of intervention are executed, including:

[0091] S301, taking the driving intention with the highest corresponding probability in the probability distribution of the driving intention as a second driving intention;

[0092] S302, when the risk warning index is less than the preset risk threshold, controlling the engine or transmission of the vehicle according to the second driving intention;

[0093] S303, when the risk warning index is greater than or equal to the preset risk threshold, controlling the brake and power system of the vehicle according to the second driving intention.

[0094] It can be understood that the engine or transmission is mainly controlled for manual driving, and the manual driving is assisted to achieve the goal of smooth driving. The brake and power system of the vehicle is mainly controlled for forced braking or starting to achieve the goal of emergency avoidance.

[0095] For example, two levels of intervention strategies are divided, which are the gentle acceleration and deceleration intervention strategy and the forced acceleration and deceleration intervention strategy.

[0096] Gentle acceleration and deceleration intervention: when the risk is low or moderate, and the driver's intention is normal but needs to be fine-tuned in the environment. For example, when the weather is sunny, the surrounding environment is good, and there is a slight congestion trend in front of the road, the system controls the engine throttle opening or transmission shift logic to gently accelerate and decelerate, improve the smoothness and comfort of driving, and reduce energy consumption and risk.

[0097] Forced acceleration and deceleration intervention: started when the risk is high, such as sudden illness of the driver, extreme bad weather, serious danger around the vehicle (front-end chain collision), etc. The system directly controls the brake and power system, forces emergency braking, rapid deceleration or acceleration to avoid danger when necessary, and reminds the driver and surrounding vehicles through sound and light alarm.

[0098] When controlling the brake and power system of the vehicle, its manufacturing precision and reliability should be ensured. For example, the brake actuator should be able to quickly respond to system instructions and accurately control the brake force. After the driving intention grading intervention system is manufactured, it needs to be strictly debugged and tested, and the intervention effect of the system is verified and optimized through simulation of various driving scenarios and fault conditions, to ensure that the forced deceleration or gentle deceleration intervention measures can be safely and effectively executed in actual use.

[0099] The technical solution in the embodiment combines the environmental data around the vehicle, the health state data of the driver, the weather data and the operation data of the driver to analyze the current driving risk and the real intention of the driver, sets a consistency score to correct the analysis of the driving intention, and finally performs different degrees of intervention according to the current driving risk, keeps the vehicle stable driving in a low-risk state, and forces into emergency avoidance in a high-risk state. Through the technical solution of the embodiment, on the one hand, the problem that the existing driving assistance system only relies on a few factors to make decisions is solved, and through the way of comprehensive analysis of multiple factors, the driving scene and the intention of the driver are more accurately judged, and a more reasonable driving intervention strategy is formulated. On the other hand, the current situation of incomplete monitoring of the driver state, easy interference and lagging intervention is changed, the health state of the driver is comprehensively monitored, not only including the fatigue state, but also covering other health factors affecting driving safety, and according to the multi-factor analysis result, the driving process is intervened in the early stage, and the driving safety is improved. In addition, the defect that the factors are independent of each other in the prior art and are not fully integrated to cause single and inaccurate intervention strategies is made up, and through combination of different factors and actual needs, accurate forced deceleration or gentle deceleration and acceleration hierarchical intervention is realized, and the complex and changeable driving scene is adapted.

[0100] Based on the same inventive concept, the embodiment of the present application also provides a multi-factor-based driving intention hierarchical intervention system. Figure 4 The structure block diagram of the multi-factor-based driving intention hierarchical intervention system provided by the embodiment of the present application is shown in Figure 4 The system specifically includes:

[0101] The first output unit 100 is used for acquiring the first type of factor data in the current driving scene, inputting the first type of factor data into a pre-set risk warning model, and outputting a risk warning index;

[0102] The second output unit 200 is used for acquiring the second type of factor data in the current driving scene, inputting the first type of factor data and the second type of factor data into a pre-set intention recognition model, and outputting a probability distribution of the driving intention and a consistency score;

[0103] The intention correction unit 300 is used for correcting the probability distribution of the driving intention until the consistency score is greater than or equal to the pre-set score threshold when the consistency score is lower than the pre-set score threshold, and performing different levels of intervention strategies based on the risk warning index and the corrected probability distribution of the driving intention.

[0104] It should be noted that the driving intention hierarchical intervention system accesses the central processor of the vehicle and communicates through the data transmission line. The central processor of the vehicle and the data transmission line have high-speed processing and transmission capabilities. In the manufacturing process, high-performance chips and advanced circuit design are used to optimize the data transmission bus, such as using a high-speed CAN bus or a FlexRay bus, to ensure that multi-source sensor data can be quickly and accurately transmitted to the central processor for fusion and analysis. At the same time, the data processing software is optimized and integrated, and a modular design is adopted to facilitate later software upgrading and maintenance to adapt to the continuous development of algorithm and function requirements.

[0105] Based on the same inventive concept, the embodiment of the present application also provides an electronic device. Figure 5 The structure block diagram of an electronic device provided by the embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the electronic device provided by the embodiment of the present application includes one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the multi-factor-based driving intention hierarchical intervention method in any of the above embodiments. The one or more I / O interfaces 103 are connected between the processor and the memory and are configured to realize the information interaction between the processor and the memory. Figure 5

[0106] The processor 101 is a device with data processing capability, including but not limited to a central processing unit (CPU) and the like. The memory 102 is a device with data storage capability, including but not limited to a random access memory (RAM, more specifically SDRAM, DDR, etc.), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), and a flash memory (FLASH). The I / O interface (read-write interface) 103 is connected between the processor 101 and the memory 102 and can realize the information interaction between the processor 101 and the memory 102, including but not limited to a data bus (BUS) and the like.

[0107] In some embodiments, the processor 101, the memory 102, and the I / O interface 103 are connected to each other through a bus 104 and are further connected to other components of the computing device.

[0108] In some embodiments, the one or more processors 101 include a field programmable gate array.

[0109] The embodiment of the present application also provides a computer readable medium. The computer readable medium stores a computer program, wherein the program is executed by a processor to implement the steps in the multi-factor-based driving intention hierarchical intervention method in any of the above embodiments. The computer readable storage medium can be a volatile or non-volatile computer readable storage medium.​

[0110] The embodiment of the present application also provides a computer program product, comprising computer readable code or a non-volatile computer readable storage medium carrying computer readable code, when the computer readable code is run in a processor of an electronic device, the processor in the electronic device executes the above-mentioned multi-factor-based driving intention hierarchical intervention method.

[0111] Those of ordinary skill in the art understand that all or some of the steps in the method disclosed above, the functions of the modules / units in the system, the device can be implemented as software, firmware, hardware and appropriate combinations thereof. In the hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component can have multiple functions, or one function or step can be performed by several physical components in cooperation. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer readable storage medium, which can include computer storage media (or non-transitory media) and communication media (or transitory media).

[0112] As known to those of ordinary skill in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable program instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disc read only memory (CD-ROM), digital versatile disc (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, it is known to those of ordinary skill in the art that communication media typically includes computer readable program instructions, data structures, program modules or other data in modulated data signals such as carrier waves or other transport mechanisms, and can include any information delivery medium.

[0113] The computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0114] Computer readable program instructions for carrying out operations of the present application can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0115] The computer program product described herein can be embodied specifically by hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied specifically as a computer storage medium, and in another alternative embodiment, the computer program product is embodied specifically as a software product, such as a software development kit (SDK) or the like.

[0116] The computer program product described herein can be embodied specifically by hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied specifically as a computer storage medium, and in another alternative embodiment, the computer program product is embodied specifically as a software product, such as a software development kit (SDK) or the like.

[0117] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data storage cycles that change state. The instructions can be executed by one or more processors of a computer, other programmable data processing apparatus, or other devices to produce a computer-implemented process such that the instructions which execute via the one or more processors of the computer or other programmable data processing devices create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0118] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0119] The flow diagrams and the block diagrams in the drawings are presented to illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to the present application. In this regard, each block in the flow diagrams and the block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions ("instructions"). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and

[0120] Example embodiments have been disclosed herein and, although specific terms are employed, they are used in a generic and descriptive sense only and not for purposes of limitation. In some instances, it will be apparent to those skilled in the art that features, characteristics or aspects described in relation to one embodiment can be applied to other embodiments, unless otherwise clearly claimed. Accordingly, it will be understood that various changes in form and details can be made without departing from the scope of the present application as set forth in the appended claims.

Claims

1. A multi-factor-based hierarchical intervention method for driving intention, characterized in that, include: Acquire the first type of factor data under the current driving scenario, input the first type of factor data into the pre-set risk warning model, and output the risk warning index; Acquire the second type of factor data in the current driving scenario, and input the first type of factor data and the second type of factor data into a pre-set intention recognition model to output the probability distribution of driving intention and consistency score; When the consistency score is lower than a preset score threshold, the probability distribution of the driving intention is corrected until the consistency score is greater than or equal to the preset score threshold; and based on the risk warning index and the corrected probability distribution of the driving intention, different levels of intervention strategies are executed.

2. The method according to claim 1, characterized in that, The first category of factor data includes: environmental data surrounding the vehicle, driver's health status data, and weather data; The second category of factor data includes: driver's operation data.

3. The method according to claim 1, characterized in that, The risk warning model is pre-configured with a first weighting strategy, which is used to assign a fixed weight to various factor data in the first type of factor data. The risk warning model obtains a risk warning index by weighting and summing the weights of various factor data.

4. The method according to claim 1, characterized in that, The intent recognition model is pre-configured with a second weighting strategy; the second weighting strategy is used to dynamically assign weights to the first type of factor data and the second type of factor data for each driving intent; The intent recognition model obtains the probability of each driving intent by weighting and summing the weights of various factor data.

5. The method according to claim 4, characterized in that, The intent recognition model also includes a pre-configured scoring mapping table; the scoring mapping table stores scores between different risk warning indices and driving intentions of different probabilities. The driving intention with the highest probability in the probability distribution of the driving intentions is taken as the first driving intention; Based on the risk warning index output by the risk warning model, the first driving intention, and the probability of the first driving intention, the consistency score is obtained by looking up the scoring mapping table.

6. The method according to claim 1, characterized in that, The types of driving intentions include at least: smooth driving, accelerating to overtake, decelerating to avoid obstacles, emergency braking, and abnormal operation.

7. The method according to claim 1, characterized in that, Based on the aforementioned risk warning index and the corrected probability distribution of driving intentions, different levels of intervention strategies are implemented, including: The driving intention with the highest probability in the probability distribution of the driving intentions is taken as the second driving intention; When the risk warning index is less than the preset risk threshold, the vehicle's engine or transmission is controlled according to the second driving intention; When the risk warning index is greater than or equal to the preset risk threshold, the vehicle's braking and power systems are controlled according to the second driving intention.

8. A multi-factor-based hierarchical intervention system for driving intention, characterized in that, The system is configured to implement the method according to any one of claims 1-7, the system comprising: The first output unit is used to acquire the first type of factor data under the current driving scenario, input the first type of factor data into the pre-set risk warning model, and output the risk warning index; The second output unit is used to acquire the second type of factor data in the current driving scenario, and input the first type of factor data and the second type of factor data into the pre-set intention recognition model to output the probability distribution of driving intention and consistency score. The intention correction unit is used to correct the probability distribution of the driving intention until the consistency score is greater than or equal to the preset score threshold when the consistency score is lower than the preset score threshold; and to execute different levels of intervention strategies based on the risk warning index and the corrected probability distribution of the driving intention.

9. An electronic device, characterized in that, The electronic device includes: At least one processor, and a memory communicatively connected to said at least one processor; The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to perform the steps of the method according to any one of claims 1-7.

Images