Automatic driving intervention discrimination method and system based on multi-point touch sensing

Abstract

The invention provides an automatic driving intervention discrimination method and system based on multi-point touch sensing. The method comprises the following steps: collecting multi-point touch data of a driver; processing the multi-point tactile data, and calculating driving behavior prediction, state monitoring and accident risk prediction of the driver through an artificial intelligence algorithm to obtain automatic driving intervention discrimination; and providing a preset automatic driving system mode service according to the automatic driving intervention judgment. On the premise of protecting user privacy, driving safety is guaranteed, driver experience is improved, the method adapts to automatic driving systems in different stages, and the method serves as a human-oriented automatic driving transition technology and is high in practicability. The method can be widely applied to the technical field of automobile auxiliary driving.

Description

Technical Field

[0001] The present invention belongs to the technical field of automobile assisted driving, and specifically relates to an automatic driving intervention determination method and system based on multi-point tactile sensing. Background Art

[0002] With the rapid development of artificial intelligence and the growing needs of people's lives, autonomous driving has become an indispensable technology for automobiles. However, as autonomous driving technology gradually enters the L3 stage from L2, the timing and degree of autonomous driving intervention need to be solved urgently, which seriously affects the driver's driving experience and safety. According to relevant statistics, the human-machine interaction contradiction between the autonomous driving system and the driver is one of the important causes of modern road traffic accidents, and its proportion cannot be ignored.

[0003] Most of the existing methods for determining autonomous driving intervention have limitations in different aspects. Most methods mainly analyze the overall vehicle movement based on vehicle parameters, lack the processing of human factors, and lead to conflicts in human-machine interaction; most methods are completely targeted at L2 autonomous driving technology, only provide auxiliary prompts to drivers, and lack the ability to adapt to different levels of autonomous driving under the process of autonomous driving technology improvement. There is still room for exploration of autonomous driving intervention determination methods that optimize human-machine interaction and are adaptable in multiple stages.

[0004] At the same time, although some special vehicles can currently achieve the same function by monitoring the driver through a camera, camera monitoring will cause privacy leakage problems, making the driver uncomfortable and difficult to use for ordinary private vehicles. The method proposed in the present invention can solve the above problems to a certain extent and is applicable to more types of vehicles. Summary of the invention

[0005] The purpose of the present invention is to propose an automatic driving intervention judgment method and system based on multi-point tactile sensing, which can ensure driving safety and improve the driver experience while protecting user privacy.

[0006] To achieve the above object, the technical solution of the present invention is as follows:

[0007] The present invention proposes an automatic driving intervention determination method based on multi-point tactile sensing, which specifically includes the following steps:

[0008] S1. collecting multi-point tactile data of the driver and processing the multi-point tactile data;

[0009] S2. The processed multi-point tactile data is used to calculate the driver's driving behavior prediction, state monitoring, and accident risk prediction through an artificial intelligence algorithm to obtain the automatic driving intervention judgment;

[0010] S3. Provide the pre-set automatic driving system mode service according to the determination of automatic driving intervention.

[0011] Preferably, the S1 specifically includes:

[0012] Collect at least multi-point tactile data of the driver on the steering wheel and pedals, and deploy according to the actual situation of the vehicle; wherein, the multi-point tactile data includes kinesthetic signals and tactile signals; the kinesthetic signals include force, position, and torque; the tactile signals include friction, temperature, and humidity, and the force includes normal force and tangential force;

[0013] Preprocess the multi-point tactile data with different times, different positions, and different dimensions to obtain a first input vector; wherein, the preprocessing includes exponential smoothing in time and combination in spatial position;

[0014] Input the first input vector into a first feature network to obtain a first output vector; wherein, the first feature network is used to extract the temporal features of the multi-point tactile data;

[0015] Input the first output vector into a second feature network to obtain a second output vector; wherein, the second feature network is used to extract the spatial features of the multi-point tactile data.

[0016] Preferably, the first feature network sequentially includes 3 groups of stacked feature extraction units and two residual blocks, and the feature extraction unit sequentially includes a convolutional layer Conv2D, a batch normalization layer BN, and a RELU activation function;

[0017] The second feature network sequentially includes a Stem block and multiple downsampling units. The second feature network also includes multiple Stage blocks, and the outputs of the Stem block and each downsampling unit are output with different-scale features through the Stage blocks; the Stem block sequentially includes 2 groups of the feature extraction units; the Stage block divides the input features into two paths, one of which is subjected to residual operation through a residual block and then spliced with the other path of features, and then the features are extracted through the feature extraction unit and the corresponding-scale features are output.

[0018] Preferably, the S2 specifically includes:

[0019] Input the second output vector into a first detection network to output driver behavior prediction data; wherein, the driver behavior data at least includes the left and right turning behaviors of the vehicle; the first detection network generates a driver behavior prediction code based on the input vector;

[0020] Input the second output vector into a second detection network to output driver status data; wherein, the driver status data encoding includes normal, sluggish, nervous, seriously ill, and other abnormal states, the sluggishness includes a sudden slowdown in reaction caused by physiological or psychological reasons, the nervous state may be caused by physiological or psychological factors, and the other abnormal states include drunk driving; the second detection network generates a driver status encoding based on the input vector;

[0021] Input the second output vector into a third detection network to output driving accident risk data; wherein, the driving accident risk data quantifies the risk prediction of whether an accident is about to occur in the form of a floating-point value in percentage system; the third detection network generates a driving accident risk confidence level based on the input vector.

[0022] Preferably, the first detection network processes different-scale features output by the second feature network through multiple detection heads. Each detection head sequentially includes 2 groups of feature extraction units, an adaptive average pooling layer, a flattening layer, a fully connected layer, and a normalization layer. The feature extraction units sequentially include a convolutional layer Conv2D, a batch normalization layer BN, and a RELU activation function; after linearly weighting the output features of all detection heads, the network final output is obtained through the normalization layer;

[0023] The network structure of the second detection network is the same as that of the first detection network;

[0024] The third detection network processes different-scale features output by the second feature network through multiple detection heads. Each detection head sequentially includes 2 groups of feature extraction units, an adaptive average pooling layer, a flattening layer, a fully connected layer, and a Sigmoid activation function; after linearly weighting the output features of all detection heads, the network final output is obtained through the Sigmoid activation function.

[0025] Preferably, the S3 specifically includes:

[0026] Determine the current in-vehicle autonomous driving system stage and pre-select an autonomous driving intervention mode; wherein, the autonomous driving system stage includes the L1 stage of assisted driving, the L2 stage of partial autonomous driving, the L3 stage of conditional autonomous driving, and the L4 stage of highly autonomous driving; the autonomous driving intervention mode includes three modes. If the autonomous driving system stage is in the L1 stage, use partial mode 1 for intervention discrimination according to requirements. If the autonomous driving system stage is in the L2 or L3 stage, use partial mode 2 for intervention discrimination according to requirements. If the autonomous driving system stage is higher than the L3 stage, use partial mode 3 for intervention discrimination when the autonomous driving system stage is in the L3 stage under the requirements of laws and regulations;

[0027] The discrimination of autonomous driving intervention for the above mode selection is based on driving behavior prediction data, driver status data, and driving accident risk data.

[0028] Preferably, the mode 1 includes the following optional embodiments:

[0029] 1) Maintain the limited auxiliary takeover intervention of the autonomous driving system; the limited auxiliary takeover intervention includes the turn signal switch based on the driving behavior prediction data. Specifically, when the driving behavior prediction data is a left turn or a right turn, the left turn signal or the right turn signal of the vehicle is correspondingly turned on, and turned off at other times;

[0030] If the driver status data is sluggish, store the data at the current moment to the reserved unit of the multi-point tactile data acquisition device for monitoring;

[0031] If the driver status data is nervous, store the data at the current moment to the reserved unit of the multi-point tactile data acquisition device for monitoring;

[0032] If the driver status data is acute illness and other abnormal conditions, store the data at the current moment to the reserved unit of the multi-point tactile data acquisition device for monitoring;

[0033] If the driving accident risk data is higher than the preset danger threshold, store the data at the current moment to the reserved unit of the multi-point tactile data acquisition device;

[0034] 2) If the driver status data is sluggish, the autonomous driving system intervenes by voice to prompt for a rest;

[0035] 3) If the driver status data is nervous, the autonomous driving system intervenes by voice to provide driving suggestions for this section of the road;

[0036] 4) If the driver status data is acute illness and other abnormal conditions, or the driving accident risk data is higher than the preset danger threshold, the autonomous driving system intervenes by voice to ask the driver whether to accept the emergency takeover intervention of the autonomous driving system. If there is no feedback or consent, then the limited emergency takeover intervention of the autonomous driving system is carried out; among them, the limited emergency takeover intervention includes intermittent honking, turning on the hazard lights, online calling for help, and emergency medical contact.

[0037] Preferably, the mode 2 includes the following optional embodiments:

[0038] 1) Maintain the limited auxiliary takeover intervention of the autonomous driving system; the limited auxiliary takeover intervention includes the turn signal switch based on the driving behavior prediction data. Specifically, when the driving behavior prediction data is a left turn or a right turn, the left turn signal or the right turn signal of the vehicle is correspondingly turned on, and turned off at other times;

[0039] 2) The driver takes over and intervenes with the assistance of the automatic driving system according to the required request; the said assisted takeover intervention includes the automatic driving system judging the driving behavior prediction data, specifically for fine steering control and fine speed control;

[0040] If the driver status data is sluggish, store the data at the current moment into the reserved unit of the multi-point tactile data acquisition device for monitoring;

[0041] If the driver status data is nervous, store the data at the current moment into the reserved unit of the multi-point tactile data acquisition device for monitoring;

[0042] If the driver status data is acute illness and other abnormal states, store the data at the current moment into the reserved unit of the multi-point tactile data acquisition device for monitoring;

[0043] If the driving accident risk data is higher than the preset danger threshold, store the data at the current moment into the reserved unit of the multi-point tactile data acquisition device;

[0044] 3) If the driver status data is sluggish, the automatic driving system intervenes by voice to prompt for rest;

[0045] 4) If the driver status data is nervous, the automatic driving system intervenes by voice to provide driving suggestions for this section of the road;

[0046] 5) If the driver status data is acute illness and other abnormal states, or the driving accident risk data is higher than the preset intervention threshold, the automatic driving system intervenes by voice to ask the driver whether to accept the emergency takeover intervention of the automatic driving system. If there is no feedback or consent, then the emergency takeover intervention of the automatic driving system is carried out; among them, the said emergency takeover intervention includes reasonable pulling over to the side of the road, turning on the hazard lights, online calling for help and emergency medical contact under the existing laws and regulations;

[0047] 6) If the driver status data is sluggish, nervous or acute illness and other abnormal states, and the emergency takeover intervention of the automatic driving system has not been carried out, and if the driving accident risk data is higher than the preset danger threshold, the automatic driving system intervenes by voice and conducts multi-faceted emergency takeover intervention of the automatic driving system; among them, the said multi-faceted emergency takeover intervention includes adjusting the steering degree, acceleration and deceleration degree, emergency braking or spot braking, turning on and off the indicating lights, automatic cruise and automatic parking based on the driving behavior prediction data under the existing laws and regulations;

[0048] 7) If in the state of autonomous driving emergency takeover intervention or multi - aspect emergency takeover intervention, and if the driver status data remains in a sluggish, nervous, seriously ill or other abnormal state within the preset recovery time threshold, continuously conduct voice intervention of the autonomous driving system and multi - aspect emergency takeover intervention of the autonomous driving system; wherein, the multi - aspect emergency takeover intervention includes adjustments to the steering degree, acceleration and deceleration degree, emergency braking or light braking, switching of indicating lights, automatic cruise and automatic parking based on driving behavior prediction data under existing laws and regulations;

[0049] If in the state of autonomous driving emergency takeover intervention and the driver status data changes from a sluggish, nervous, seriously ill or other abnormal state to normal, conduct autonomous driving voice intervention to ask whether the driver's permission needs to be returned;

[0050] If in the state of autonomous driving emergency takeover intervention or multi - aspect emergency takeover intervention, the driver status data returns to normal and the driving accident risk data is lower than the preset intervention threshold, conduct autonomous driving voice intervention to prompt the return of the driver's permission;

[0051] If in the state of autonomous driving emergency takeover intervention or multi - aspect emergency takeover intervention, the driver status data returns to normal, the driving accident risk data is higher than the preset intervention threshold and lower than the preset danger threshold, conduct autonomous driving voice intervention to ask whether the driver's permission needs to be returned.

[0052] Preferably, the mode 3 includes the following optional implementation manners:

[0053] 1) Maintain the limited - assistance takeover intervention of the autonomous driving system; the limited - assistance takeover intervention includes the switching of indicating lights based on driving behavior prediction data. Specifically, if the driving behavior prediction data is a left turn or a right turn, turn on the left turn signal or the right turn signal of the vehicle accordingly, and turn them off at other times;

[0054] 2) The driver requests the autonomous driving system to assist in takeover intervention according to the requirements; the assistance takeover intervention includes the autonomous driving system's judgment of driving behavior prediction data, specifically fine control of steering and fine control of speed;

[0055] If the driver status data is sluggish, store the data at the current moment in the reserved unit of the multi - point tactile data acquisition device for monitoring;

[0056] If the driver status data is nervous, store the data at the current moment in the reserved unit of the multi - point tactile data acquisition device for monitoring;

[0057] If the driver status data is seriously ill or in other abnormal states, store the data at the current moment in the reserved unit of the multi - point tactile data acquisition device for monitoring;

[0058] If the driving accident risk data is higher than the preset danger threshold, store the data at the current moment in the reserved unit of the multi-point tactile data acquisition device;

[0059] 3) If the driver status data is dull, the autonomous driving system intervenes by voice to prompt for rest;

[0060] 4) If the driver status data is nervous, the autonomous driving system intervenes by voice to provide driving suggestions for this section of the road;

[0061] 5) If the driver status data is acute illness and other abnormal states, the autonomous driving system intervenes by voice to ask the driver whether to accept part of the discrimination of the emergency takeover intervention of the autonomous driving system. If there is no feedback or consent, then perform part of the discrimination of the emergency takeover intervention of the autonomous driving system; among them, the part of the emergency takeover intervention includes online calling for help and emergency medical contact;

[0062] 6) If the driver status data is acute illness and other abnormal states, or the driving accident risk data is higher than the preset intervention threshold, the autonomous driving system intervenes by voice to ask the driver whether to accept the full takeover intervention of the autonomous driving system. If there is no feedback or consent, then perform the full takeover intervention of the autonomous driving system; among them, the full takeover intervention includes the control of the whole vehicle under the existing laws and regulations;

[0063] If the driver status data is dull, nervous or acute illness and other abnormal states, and the full takeover intervention of the autonomous driving system has not been performed, and if the driving accident risk data is higher than the preset danger threshold, perform the full takeover intervention of the autonomous driving system; among them, the full takeover intervention includes the control of the whole vehicle under the existing laws and regulations;

[0064] 7) If in the state of full takeover intervention of autonomous driving, if the driver status data is still dull, nervous or in acute illness and other abnormal states within the preset recovery time threshold, continue to perform the full takeover intervention of the autonomous driving system; among them, the full takeover intervention includes the control of the whole vehicle under the existing laws and regulations;

[0065] If in the state of full takeover intervention of autonomous driving and the driver status data changes from dull, nervous or acute illness and other abnormal states to normal, the autonomous driving voice intervenes to ask whether the driver's permission needs to be returned;

[0066] If the driver status data returns to normal and the driving accident risk data is lower than the preset danger threshold in the state of full takeover intervention of autonomous driving, the autonomous driving voice intervenes to ask whether the driver's permission needs to be returned.

[0067] The present invention also proposes an autonomous driving intervention discrimination system based on multi-point tactile sensing. The system is implemented by using any one of the above-mentioned autonomous driving intervention discrimination methods based on multi-point tactile sensing, and includes:

[0068] The acquisition unit is deployed at least on the surface of the steering wheel and the pedal surface, and at least acquires multi-point tactile data of the driver on the steering wheel and the pedal, and is deployed according to the actual situation of the vehicle; wherein, the multi-point tactile data includes kinesthetic and vibrotactile signals; the kinesthetic signals include force, position, and torque; the vibrotactile signals include friction, temperature, and humidity, wherein the force includes normal force and tangential force; the deployment should at least meet that there are no less than 32 acquisition points on the surface of the steering wheel and no less than 16 acquisition points on the surface of each pedal;

[0069] The storage unit is used to store the multi-point tactile data at some moments, and stores and releases data in real time as the moment changes;

[0070] The analysis unit is used to analyze the multi-point tactile data to obtain driving behavior prediction results, driver state analysis results, and driving accident risk prediction results;

[0071] The reserved unit is used to store the multi-point tactile data at specific moments; the specific moments include the moments when the accident prediction exceeds the threshold and the moments when the driver's state is abnormal.

[0072] Compared with the prior art, the present invention has the following beneficial effects:

[0073] The human-machine interaction concept with human as the first element completely revolves around the driver himself. All data is collected in a natural way without relying on other non-vehicle-mounted devices; all decisions are analyzed with the driving experience as the core without deviating from the driver's intention;

[0074] Data usage that protects personal privacy, does not use cameras, completely abandons the video data and eye movement data during the driver's driving, and only uses the driver's multi-point tactile data;

[0075] It adapts to different stages of the autonomous driving system, can select the intervention degree of the autonomous driving system to take over, and can be widely deployed in the long term as a general transition technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 It is a schematic diagram of the functional modules of a multi-point tactile data acquisition device in an embodiment of the present invention;

[0077] Figure 2 It is a schematic flow diagram of a multi-point tactile data feature analysis method in an embodiment of the present invention;

[0078] Figure 3 It is a schematic flow diagram of a driving behavior prediction method in an embodiment of the present invention;

[0079] Figure 4 It is a schematic flow diagram of a driving behavior prediction method in an embodiment of the present invention;

[0080] Figure 5 This is a schematic flowchart of a driving accident risk prediction method in an embodiment of the present invention;

[0081] Figure 6 This is a schematic flowchart of an autonomous driving intervention method in an embodiment of the present invention;

[0082] Figure 7 This is a network structure diagram of a first feature network and a second feature network in an embodiment of the present invention;

[0083] Figure 8 This is a network structure diagram of a first detection network and a second detection network in an embodiment of the present invention;

[0084] Figure 9 This is a network structure diagram of a third detection network in an embodiment of the present invention. Detailed implementation manners

[0085] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention. Figures 1-9 To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0086] The present invention provides a method and system for autonomous driving intervention discrimination based on multi-point tactile sensing. The method collects multi-point tactile data of a driver; processes the multi-point tactile data, and calculates the driving behavior prediction, state monitoring, and accident risk prediction of the driver through an artificial intelligence algorithm to obtain the autonomous driving intervention discrimination; and provides a pre-set autonomous driving system mode service according to the autonomous driving intervention discrimination.

[0087] In an exemplary embodiment, as shown in, a multi-point tactile data collection device is provided, including:

[0088] In an exemplary embodiment, as Figure 1 shown, a multi-point tactile data collection device is provided, including:

[0089] The acquisition unit 101 is deployed at least on the surface of the steering wheel and the pedal surface, and at least collects multi-point tactile data of the driver on the steering wheel and the pedal, and is deployed according to the actual situation of the vehicle; wherein, the multi-point tactile data includes kinesthetic and vibrotactile signals; the kinesthetic signals include force (normal force and tangential force), position, and torque; the vibrotactile signals include friction, temperature, and humidity; the deployment should at least meet that there are no less than 32 acquisition points on the surface of the steering wheel and no less than 16 acquisition points on each pedal surface; as an optional implementation manner, the acquisition unit 101 preferentially ensures the collection of the normal force and tangential force applied by the driver to the steering wheel and the pedal, and secondly collects the temperature and humidity of the driver's palm, and whether to collect the temperature and humidity of the palm can be selected according to the actual software calculation speed requirements;

[0090] The storage unit 102 is used to store multi-point tactile data at some moments, and stores and releases data in real time as the moments change; as an optional implementation manner, the storage unit 102 stores n frames of the multi-point tactile data at each moment, and releases the last frame when reading the latest frame;

[0091] The analysis unit 103 is used to analyze the multi-point tactile data to obtain driving behavior prediction results, driver state analysis results, and driving accident risk prediction results; as an optional implementation manner, the analysis unit 103 stores all networks and calculation parameters in the multi-point tactile data feature analysis method, driving behavior prediction method, driver state monitoring, and driving accident risk prediction method;

[0092] The reserved unit 104 is used to store multi-point tactile data at specific moments; the specific moments include the moments when the accident prediction exceeds the threshold and the moments when the driver's state is abnormal.

[0093] In an exemplary embodiment, as Figure 2 shown, a multi-point tactile data feature analysis method is provided. This method is executed by a computer device, and can be specifically executed by a computer device such as a terminal or a server alone, or jointly executed by a terminal and a server. In the embodiments of the present application, the following steps S201 to step S204 are included:

[0094] Step S201, collect multi-point tactile data of the driver; wherein, the multi-point tactile data includes kinesthetic and vibrotactile signals; the kinesthetic signals include force (normal force and tangential force), position, and torque; the vibrotactile signals include friction, temperature, and humidity;

[0095] Step S202: Preprocess the multi-point tactile data at different times, different positions, and different dimensions to obtain a first input vector. Among them, the preprocessing includes exponential smoothing in time and combination in spatial position. As an alternative implementation, if the multi-point tactile data at n time points is input, the process of the preprocessing is as follows:

[0096] s i = αx i +(1 - α)(s i-1 + t i-1 ),

[0097] t i = β(s i - s i-1 )+(1 - β)t i-1 ,

[0098]

[0099] Among them, x i represents the multi-point tactile data at the i-th time point, s i represents the smoothed value at the i-th time point, t i represents the difference between the smoothed value at the i-th time point and the smoothed value at the previous time point. α and β are adjustable hyperparameters of the preprocessing. s represents the first input vector, T represents the number of time points, which is n in this embodiment, and D represents the input feature dimension of each time point;

[0100] Step S203: Input the first input vector into a first feature network to obtain a first output vector. Among them, the first feature network is used to extract the temporal features of the multi-point tactile data;

[0101] Step S204: Input the first output vector into a second feature network to obtain a second output vector. Among them, the second feature network is used to extract the spatial features of the multi-point tactile data;

[0102] The network structures of the first feature network and the second feature network refer to Figure 7 , where Figure 7 the three-scale features P3, P4, and P5 output by the second feature network shown are only examples, and the actual number of feature scales can be set according to requirements.

[0103] In an exemplary embodiment, as Figure 3 shown, a driving behavior prediction method is provided. This method is executed by a computer device, and specifically can be executed alone by a computer device such as a terminal or a server, or jointly executed by a terminal and a server. In the embodiments of the present application, it includes the following steps S301 to S303:

[0104] Step S301: Obtain the second output vector, where the second output vector includes multiple type dimensions.

[0105] Step S302: Input the second output vector into the first detection network to obtain driving behavior prediction encodings corresponding to different type dimensions, where the driving behavior prediction encodings correspond to vehicle steering behaviors (left and right turns). As an alternative implementation, the calculation process of the first detection network is as follows:

[0106] h 1_i = Dect 1_i (y i ),

[0107] where Dect 1_i (·) represents the detection heads of different sizes of the second output vector corresponding to the i-th type dimension in the first detection network. The detection heads include convolutional layers, activation functions, and batch normalization layers with adjustable numbers and parameters. h 1_i represents the driving behavior prediction encoding, and y i represents the second output vector. Among them, the driving behavior prediction encoding includes [1, 0] and [0, 1], corresponding to vehicle steering behaviors (left and right turns) as required.

[0108] Step S303: Integrate the driving behavior prediction encodings output by the detection heads of different sizes in the first detection network to output the final driving behavior prediction data. As an alternative implementation, the integration process is as follows:

[0109] h 1 = ∑w 1_i h 1_i , ∑w 1_i = 1,

[0110] where h 1 represents the final driving behavior data, and w 1_i is the weight parameter. The final driving behavior prediction data includes vehicle steering behaviors (left and right turns).

[0111] The network structure of the first detection network refers to Figure 8 ; As an alternative implementation, individual detection heads can be specified to be retained and the calculation processes of other corresponding sizes in the first feature network, second feature network, and first detection network can be discarded.

[0112] In an exemplary embodiment, such as Figure 4As shown, a driver state monitoring method is provided. This method is executed by a computer device, which can be specifically executed by a computer device such as a terminal or a server alone, or jointly executed by a terminal and a server. In the embodiments of the present application, it includes the following steps S401 to S403.

[0113] Step S401, obtain a second output vector; wherein, the second output vector includes multiple type-size forms;

[0114] Step S402, input the second output vector into a second detection network to obtain driver state encodings corresponding to different type-sizes; wherein, the driver state encodings correspond to normal, dull, nervous, seriously ill, and other abnormal states; As an alternative implementation, the calculation process of the second detection network is as follows:

[0115] h 2_i = Dect 2_i (y i ),

[0116] wherein, Dect 2_i (·) represents the detection heads of different sizes of the second output vector corresponding to the i type-size in the second detection network. The detection heads include convolutional layers, activation functions, and batch normalization layers with adjustable quantities and parameters. h 2_i represents the driver state encoding; As an alternative implementation, since the multi-point tactile data feature analysis results of seriously ill and other abnormal states are similar, they are classified into the same category; wherein, the driver state encodings include [1,0,0,0], [0,1,0,0], [0,0,1,0], and [0,0,0,1], corresponding to normal, dull, nervous, seriously ill, and other abnormal states as required;

[0117] Step S403, integrate the driver state encodings output by the detection heads of different sizes in the second detection network to output the final driver state data; As an alternative implementation, the integration process is as follows:

[0118] h 2 = ∑w 2_i h 2_i , ∑w 2_i = 1,

[0119] wherein, h 2 represents the final driver state data, and w 2_i is a weight parameter; the final driver state data includes normal, dull, nervous, seriously ill, and other abnormal states;

[0120] The network structure of the second detection network refers to Figure 8; As an alternative implementation, it is possible to specify that individual detection heads are retained and the calculation processes for other corresponding sizes in the first feature network, the second feature network, and the second detection network are discarded.

[0121] In an exemplary embodiment, as Figure 5 shown, a driving accident risk prediction method is provided. This method is executed by a computer device, specifically, it can be executed independently by a computer device such as a terminal or a server, or jointly executed by a terminal and a server. In the embodiments of the present application, the following steps S501 to step S503 are included.

[0122] Step S501, obtain a second output vector; wherein, the second output vector includes multiple type size forms;

[0123] Step S502, input the second output vector into a third detection network to obtain driving accident risk data corresponding to different type sizes; wherein, the driving accident risk data is presented in the form of a percentage floating-point value; As an alternative implementation, the calculation process of the third detection network is as follows:

[0124] p i = Dect 3_i (y i ),

[0125] wherein, Dect 3_i (·) represents the detection heads of different sizes of the second output vector corresponding to the i type size in the third detection network. The detection heads include convolutional layers and activation functions with adjustable quantity and parameters. p i represents the driving accident risk confidence; wherein, the driving accident risk confidence is presented in the form of a percentage floating-point value;

[0126] Step S503, weight the driving accident risk confidences output by the detection heads of different sizes in the third detection network to output the final driving accident risk data; As an alternative implementation, the integration process is as follows:

[0127] p = ∑w 3_i p i , ∑w 3_i = 1,

[0128] wherein, p represents the final driving accident risk data, and w 3_i is a weight parameter; the final driving accident risk data is presented in the form of a percentage floating-point value;

[0129] The network structure of the third detection network refers to Figure 9; As an optional implementation, it is possible to specify to retain the use of individual detection heads and discard the calculation processes of other corresponding sizes in the first feature network, the second feature network, and the third detection network.

[0130] In an exemplary embodiment, as Figure 6 shown, an autonomous driving intervention method is provided. This method is executed by a computer device, specifically, it can be executed independently by a computer device such as a terminal or a server, or jointly executed by a terminal and a server. In the embodiments of the present application, the following steps S601 to step S605 are included.

[0131] Step S601, determine the current in-vehicle autonomous driving system stage and pre-select an autonomous driving intervention mode in advance; wherein, the autonomous driving system stage includes the L1 stage of assisted driving, the L2 stage of partial autonomous driving, the L3 stage of conditional autonomous driving, and the L4 stage of highly autonomous driving; the autonomous driving intervention mode includes three modes. If the autonomous driving system stage is in L1, partial mode 1 can be used for intervention discrimination according to requirements. If the autonomous driving system stage is in L2 or L3, mode 2 can be used according to requirements. If the autonomous driving system stage is higher than the L3 stage, mode 3 can be used. Under legal requirements, partial mode 3 can be used for intervention discrimination when the autonomous driving system stage is in L3;

[0132] Step S602, collect at least multi-point tactile data of the driver on the steering wheel and pedals and deploy it according to the actual situation of the vehicle; wherein, the multi-point tactile data includes kinesthetic and vibrotactile signals; the kinesthetic signals include force (normal force and tangential force), position, and torque; the vibrotactile signals include friction, temperature, and humidity;

[0133] Step S603, obtain a second output vector from the multi-point tactile data through the multi-point tactile data feature analysis method;

[0134] Step S604, obtain the driving behavior data, driver state data, and driving accident risk data from the second output vector through the driving behavior prediction method, driver state monitoring method, and driving accident risk prediction method;

[0135] Step S605, perform autonomous driving intervention discrimination based on the driving behavior prediction data, driver state data, driving accident risk data, and the autonomous driving intervention mode, specifically as follows:

[0136] (1) Mode 1

[0137] As an alternative implementation, maintain the limited assisted takeover intervention of the autonomous driving system; the limited assisted takeover intervention includes the turn signal switch based on the driving behavior prediction data, specifically: when the driving behavior prediction data is a left (right) turn, turn on the left (right) turn signal of the vehicle, and turn it off at other times;

[0138] If the driver status data is sluggish, store the data at the current moment in the reserved unit of the multi-point tactile data acquisition device for further monitoring;

[0139] If the driver status data is nervous, store the data at the current moment in the reserved unit of the multi-point tactile data acquisition device for further monitoring;

[0140] If the driver status data is serious illness and other abnormal conditions, store the data at the current moment in the reserved unit of the multi-point tactile data acquisition device for further monitoring;

[0141] If the driving accident risk data is higher than the preset danger threshold, store the data at the current moment in the reserved unit of the multi-point tactile data acquisition device;

[0142] As an alternative implementation, if the driver status data is sluggish, the autonomous driving system intervenes with voice to prompt for rest;

[0143] As an alternative implementation, if the driver status data is nervous, the autonomous driving system intervenes with voice to provide driving suggestions for this section of the road;

[0144] As an alternative implementation, if the driver status data is serious illness and other abnormal conditions, or the driving accident risk data is higher than the preset danger threshold, the autonomous driving system intervenes with voice to ask the driver whether to accept the emergency takeover intervention of the autonomous driving system. If there is no feedback or consent, perform the limited emergency takeover intervention of the autonomous driving system; among them, the limited emergency takeover intervention includes intermittent honking, turning on the hazard lights, online calling for help, and emergency medical contact;

[0145] (2) Mode 2

[0146] As an alternative implementation, maintain the limited assisted takeover intervention of the autonomous driving system; the limited assisted takeover intervention includes the turn signal switch based on the driving behavior prediction data, specifically: when the driving behavior prediction data is a left (right) turn, turn on the left (right) turn signal of the vehicle, and turn it off at other times;

[0147] As an alternative implementation, the driver can request the assisted takeover intervention of the autonomous driving system; the assisted takeover intervention includes the further judgment of the driving behavior prediction data by the autonomous driving system, specifically the fine control of steering and the fine control of speed;

[0148] If the driver's status data is sluggish, store the data at the current moment in the reserved unit of the multi-point tactile data acquisition device for further monitoring;

[0149] If the driver's status data is nervous, store the data at the current moment in the reserved unit of the multi-point tactile data acquisition device for further monitoring;

[0150] If the driver's status data is acute illness and other abnormal states, store the data at the current moment in the reserved unit of the multi-point tactile data acquisition device for further monitoring;

[0151] If the driving accident risk data is higher than the preset danger threshold, store the data at the current moment in the reserved unit of the multi-point tactile data acquisition device;

[0152] As an alternative implementation, if the driver's status data is sluggish, the autonomous driving system intervenes by voice to prompt for rest;

[0153] As an alternative implementation, if the driver's status data is nervous, the autonomous driving system intervenes by voice to provide driving suggestions for this section of the road;

[0154] As an alternative implementation, if the driver's status data is acute illness and other abnormal states, or the driving accident risk data is higher than the preset intervention threshold, the autonomous driving system intervenes by voice to ask the driver whether to accept the emergency takeover intervention of the autonomous driving system. If there is no feedback or consent, the emergency takeover intervention of the autonomous driving system is carried out; wherein, the emergency takeover intervention includes reasonable pulling over to the side of the road, turning on the hazard lights, online calling for help and emergency medical contact under the existing laws and regulations;

[0155] As an alternative implementation, if the driver's status data is sluggish, nervous or acute illness and other abnormal states, and the emergency takeover intervention of the autonomous driving system has not been carried out, if the driving accident risk data is higher than the preset danger threshold, the voice intervention of the autonomous driving system and the multi-faceted emergency takeover intervention of the autonomous driving system are carried out; wherein, the multi-faceted emergency takeover intervention includes the adjustment of the steering degree, acceleration and deceleration degree, emergency braking or point braking, turning on and off the signal lights, automatic cruise and automatic parking based on the driving behavior prediction data under the existing laws and regulations;

[0156] As an alternative implementation, if in the state of emergency takeover intervention or multi-faceted emergency takeover intervention of the autonomous driving system, if the driver's status data is still in the state of sluggishness, nervousness or acute illness and other abnormal states within the preset recovery time threshold, the voice intervention of the autonomous driving system and the multi-faceted emergency takeover intervention of the autonomous driving system are continuously carried out; wherein, the multi-faceted emergency takeover intervention includes the adjustment of the steering degree, acceleration and deceleration degree, emergency braking or point braking, turning on and off the signal lights, automatic cruise and automatic parking based on the driving behavior prediction data under the existing laws and regulations;

[0157] If, in the state of emergency takeover intervention of autonomous driving and the driver status data changes from dullness, nervousness, acute illness, or other abnormal states to normal, autonomous driving voice intervention is performed to ask whether the driver's permission needs to be returned;

[0158] If, in the state of emergency takeover intervention of autonomous driving or multi - aspect emergency takeover intervention, the driver status data returns to normal and the driving accident risk data is lower than the preset intervention threshold, autonomous driving voice intervention is performed to prompt the return of the driver's permission;

[0159] If, in the state of emergency takeover intervention of autonomous driving or multi - aspect emergency takeover intervention, the driver status data returns to normal, the driving accident risk data is higher than the preset intervention threshold and lower than the preset danger threshold, autonomous driving voice intervention is performed to ask whether the driver's permission needs to be returned;

[0160] If, in the state of emergency takeover intervention of autonomous driving or multi - aspect emergency takeover intervention, the driver status data returns to normal, the driving accident risk data is higher than the preset intervention threshold and lower than the preset danger threshold, autonomous driving voice intervention is performed to ask whether the driver's permission needs to be returned;

[0161] (3) Mode 3

[0162] As an alternative implementation, maintain the limited - assistance takeover intervention of the autonomous driving system; the limited - assistance takeover intervention includes the switch of the indicating lamp based on the driving behavior prediction data. Specifically, if the driving behavior prediction data is for a left (right) turn, turn on the left (right) turn signal of the vehicle, and turn it off at other times;

[0163] As an alternative implementation, the driver can request the autonomous driving system to assist in takeover intervention; the assistance takeover intervention includes further judgment of the driving behavior prediction data by the autonomous driving system, specifically, fine control of steering and fine control of speed;

[0164] If the driver status data is dull, store the data at the current moment in the reserved unit of the multi - point tactile data acquisition device for further monitoring;

[0165] If the driver status data is nervous, store the data at the current moment in the reserved unit of the multi - point tactile data acquisition device for further monitoring;

[0166] If the driver status data is acute illness or other abnormal states, store the data at the current moment in the reserved unit of the multi - point tactile data acquisition device for further monitoring;

[0167] If the driving accident risk data is higher than the preset danger threshold, store the data at the current moment in the reserved unit of the multi - point tactile data acquisition device;

[0168] As an alternative implementation, if the driver status data indicates sluggishness, the autonomous driving system intervenes with voice prompts to suggest taking a break;

[0169] As an alternative implementation, if the driver status data indicates nervousness, the autonomous driving system intervenes with voice prompts to provide driving suggestions for this section of the road;

[0170] As an alternative implementation, if the driver status data indicates acute illness or other abnormal conditions, the autonomous driving system intervenes with voice prompts to ask the driver whether to accept partial discrimination for emergency takeover intervention by the autonomous driving system. If there is no feedback or consent, partial discrimination for emergency takeover intervention by the autonomous driving system is performed; among them, the partial discrimination for emergency takeover intervention includes online calling for help and emergency medical contact;

[0171] As an alternative implementation, if the driver status data indicates acute illness or other abnormal conditions, or the driving accident risk data is higher than the preset intervention threshold, the autonomous driving system intervenes with voice prompts to ask the driver whether to accept full takeover intervention by the autonomous driving system. If there is no feedback or consent, full takeover intervention by the autonomous driving system is performed; among them, the full takeover intervention includes the control of the entire vehicle under existing laws and regulations;

[0172] As an alternative implementation, if the driver status data indicates sluggishness, nervousness, acute illness or other abnormal conditions, and full takeover intervention by the autonomous driving system has not been performed, and the driving accident risk data is higher than the preset danger threshold, full takeover intervention by the autonomous driving system is performed; among them, the full takeover intervention includes the control of the entire vehicle under existing laws and regulations;

[0173] As an alternative implementation, if in the state of full takeover intervention by the autonomous driving system, and the driver status data still indicates sluggishness, nervousness, acute illness or other abnormal conditions within the preset recovery time threshold, continuous full takeover intervention by the autonomous driving system is performed; among them, the full takeover intervention includes the control of the entire vehicle under existing laws and regulations;

[0174] If in the state of full takeover intervention by the autonomous driving system and the driver status data changes from sluggishness, nervousness, acute illness or other abnormal conditions to normal, the autonomous driving system intervenes with voice prompts to ask whether the driver's permissions need to be returned;

[0175] If in the state of full takeover intervention by the autonomous driving system, the driver status data returns to normal and the driving accident risk data is lower than the preset danger threshold, the autonomous driving system intervenes with voice prompts to ask whether the driver's permissions need to be returned;

[0176] As an alternative implementation, the in-vehicle autonomous driving system is pre-installed with a voice dialogue system to perform real-time mode fine-tuning for the driver's intentions.

[0177] The computer-readable storage medium provided by the above embodiments of the present invention and the method for judging autonomous driving intervention based on multi-point tactile sensing provided by the embodiments of the present invention are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0178] It should be noted that a large number of specific details are set forth in the specification provided herein. However, it is understood that the embodiments of the present invention may be practiced without these specific details. In some instances, well-known structures and technologies have not been shown in detail so as not to obscure the understanding of this specification.

[0179] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover a non-exclusive inclusion, such that a process, method, article or apparatus comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or apparatus. Without further limitation, an element defined by the phrase "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising the element.

[0180] The above are only the embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, various changes and modifications can be made to this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the scope of the claims of the present invention.

Claims

1. A method for determining automatic driving intervention based on multi-point tactile sensing, characterized in that: The specific steps include: S1. collecting multi-point tactile data of the driver and processing the multi-point tactile data; S2. The processed multi-point tactile data is used to calculate the driver's driving behavior prediction, state monitoring, and accident risk prediction through an artificial intelligence algorithm to obtain the automatic driving intervention judgment; S3. Providing a pre-set autonomous driving system mode service based on the autonomous driving intervention determination.

2. The method for determining automatic driving intervention based on multi-point tactile sensing according to claim 1, characterized in that: The S1 specifically includes: At least the driver's multi-point tactile data on the steering wheel and pedals are collected; wherein the multi-point tactile data includes kinesthetic signals and vibration signals; the kinesthetic signals include force, position, and torque; the vibration signals include friction, temperature, and humidity, wherein the force includes normal force and tangential force; Preprocessing the multi-point tactile data at different times, different positions and different dimensions to obtain a first input vector; wherein the preprocessing includes exponential smoothing in time and a combination in spatial position; Inputting the first input vector into a first feature network to obtain a first output vector; wherein the first feature network is used to extract the temporal features of the multi-point tactile data; The first output vector is input into a second feature network to obtain a second output vector; wherein the second feature network is used to extract the spatial features of the multi-point tactile data.

3. The method for determining automatic driving intervention based on multi-point tactile sensing according to claim 2, characterized in that: The first feature network sequentially includes three groups of stacked feature extraction units and two residual blocks, and the feature extraction unit sequentially includes a convolutional layer Conv2D, a batch normalization layer BN and a RELU activation function; The second feature network includes a Stem block and multiple downsampling units in sequence, and the second feature network also includes multiple Stage blocks, and the outputs of the Stem block and each of the downsampling units output features of different scales through the Stage block; the Stem block includes 2 groups of feature extraction units in sequence; the Stage block divides the input features into two paths, one of which is concatenated with the other feature after performing residual operation through the residual block, and then outputs features of corresponding scales after feature extraction through the feature extraction unit.

4. The method for determining automatic driving intervention based on multi-point tactile sensing according to claim 2, characterized in that: The S2 specifically includes: Inputting the second output vector into the first detection network to output driver behavior prediction data; wherein the driver behavior data at least includes the left and right steering behavior of the vehicle; the first detection network generates a driver behavior prediction code based on the input vector; Input the second output vector to the second detection network to output driver status data; wherein the driver status data encoding includes normal, slow, nervous, acute illness and other abnormal states, the slowness includes a sudden slow reaction caused by physiological or psychological reasons, and the other abnormal states include drunk driving; the second detection network generates the driver status encoding based on the input vector; The second output vector is input into the third detection network to output driving accident risk data; wherein the driving accident risk data quantifies the risk prediction of whether an accident is about to occur in the form of a percentage floating point value; the third detection network generates a driving accident risk confidence based on the input vector.

5. The method for determining automatic driving intervention based on multi-point tactile sensing according to claim 4, characterized in that: The first detection network processes the different scale features output by the second feature network through multiple detection heads, each detection head includes two groups of feature extraction units, an adaptive average pooling layer, a flattening layer, a fully connected layer and a normalization layer in sequence, and the feature extraction unit includes a convolutional layer Conv2D, a batch normalization layer BN and a RELU activation function in sequence; the output features of all detection heads are linearly weighted and then the final output of the network is obtained through the normalization layer; The network structure of the second detection network is the same as that of the first detection network; The third detection network processes the different scale features output by the second feature network through multiple detection heads, each detection head includes two groups of feature extraction units, an adaptive average pooling layer, a flattening layer, a fully connected layer and a Sigmoid activation function in sequence; the output features of all detection heads are linearly weighted and then the final output of the network is obtained through the Sigmoid activation function.

6. The method for determining automatic driving intervention based on multi-point tactile sensing according to claim 2, characterized in that: The S3 specifically includes: Determine the current stage of the vehicle-mounted autonomous driving system and select the autonomous driving intervention mode in advance; wherein the autonomous driving system stage includes the L1 stage of assisted driving, the L2 stage of partial autonomous driving, the L3 stage of conditional autonomous driving, and the L4 stage of highly autonomous driving; the autonomous driving intervention mode includes three modes, if the autonomous driving system stage is at the L1 stage, use partial mode 1 to intervene and judge according to demand, if the autonomous driving system stage is at the L2 or L3 stage, use partial mode 2 to intervene and judge according to demand, if the autonomous driving system stage is higher than the L3 stage, use partial mode 3 to intervene and judge when the autonomous driving system stage is at the L3 stage under the requirements of laws and regulations; The automatic driving intervention judgment of the mode selection is performed based on the driving behavior prediction data, the driver status data and the driving accident risk data.

7. The method for determining automatic driving intervention based on multi-point tactile sensing according to claim 6, characterized in that: Mode 1 includes the following optional implementations: 1) Maintaining limited assisted takeover intervention of the automatic driving system; the limited assisted takeover intervention includes switching on and off the indicator lights based on the driving behavior prediction data, specifically: if the driving behavior prediction data is to turn left or right, the left turn signal or right turn signal of the car is turned on accordingly, and turned off at other times; If the driver status data is dull, storing the current moment data to a reserved unit of the multi-point tactile data acquisition device for monitoring; If the driver status data is tense, the current moment data is stored in a reserved unit of the multi-point tactile data acquisition device for monitoring; If the driver's status data is acute illness or other abnormal status, the current moment data is stored in a reserved unit of the multi-point tactile data acquisition device for monitoring; If the driving accident risk data is higher than a preset danger threshold, storing the current moment data to a reserved unit of the multi-point tactile data acquisition device; 2) If the driver's status data is sluggish, the autonomous driving system voice intervenes to remind the driver to take a rest; 3) If the driver's status data indicates that the driver is nervous, the autonomous driving system will intervene with voice and provide driving advice for that section of the road; 4) If the driver's status data indicates acute illness or other abnormal conditions, or the driving accident risk data is higher than the preset danger threshold, the autonomous driving system will intervene by voice, asking the driver whether he or she accepts the emergency takeover of the autonomous driving system. If there is no feedback or consent, the autonomous driving system will perform limited emergency takeover; wherein, the limited emergency takeover includes intermittent honking of the horn, activation of the double flash lights, online calls for help, and emergency medical contact.

8. The method for determining automatic driving intervention based on multi-point tactile sensing according to claim 6, characterized in that: Mode 2 includes the following optional implementations: 1) Maintaining limited assisted takeover intervention of the automatic driving system; the limited assisted takeover intervention includes switching on and off the indicator lights based on the driving behavior prediction data, specifically: if the driving behavior prediction data is to turn left or right, the left turn signal or right turn signal of the car is turned on accordingly, and turned off at other times; 2) The driver requires the automatic driving system to assist in taking over the intervention according to the needs; the auxiliary takeover intervention includes the automatic driving system judging the driving behavior prediction data, specifically steering fine control and speed fine control; If the driver status data is dull, storing the current moment data to a reserved unit of the multi-point tactile data acquisition device for monitoring; If the driver status data is tense, the current moment data is stored in a reserved unit of the multi-point tactile data acquisition device for monitoring; If the driver's status data is acute illness or other abnormal status, the current moment data is stored in a reserved unit of the multi-point tactile data acquisition device for monitoring; If the driving accident risk data is higher than a preset danger threshold, storing the current moment data to a reserved unit of the multi-point tactile data acquisition device; 3) If the driver's status data is sluggish, the automatic driving system voice intervention will prompt the driver to take a rest; 4) If the driver's status data indicates that the driver is nervous, the autonomous driving system will intervene with voice and provide driving advice for that section of the road; 5) If the driver's status data indicates acute illness or other abnormal conditions, or the driving accident risk data is higher than the preset intervention threshold, the autonomous driving system intervenes with voice, asking the driver whether to accept the emergency takeover intervention of the autonomous driving system. If there is no feedback or consent, the autonomous driving system takes emergency takeover intervention; wherein, the emergency takeover intervention includes reasonable side parking, turning on the double flash lights, online call for help and emergency medical contact under existing laws and regulations; 6) If the driver's state data is dull, nervous, or acutely ill and other abnormal states, and the automatic driving system has not been taken over for emergency intervention, if the driving accident risk data is higher than the preset danger threshold, the automatic driving system voice intervention and the automatic driving system multi-faceted emergency takeover intervention are carried out; wherein, the multi-faceted emergency takeover intervention includes the adjustment of the steering degree, acceleration and deceleration degree, emergency braking or braking, the switch of the indicator light, automatic cruising and automatic parking based on the driving behavior prediction data under the existing laws and regulations; 7) If the driver's status data is still in a state of dullness, tension, acute illness or other abnormal state within the preset recovery time threshold during the automatic driving emergency takeover intervention or multi-faceted emergency takeover intervention, the automatic driving system voice intervention and the automatic driving system multi-faceted emergency takeover intervention are continued; wherein, the multi-faceted emergency takeover intervention includes the adjustment of the steering degree, acceleration and deceleration degree, emergency braking or braking, the switch of the indicator light, automatic cruising and automatic parking based on the driving behavior prediction data under the existing laws and regulations; If the driver is in an emergency autopilot takeover state and the driver's status data changes from dullness, tension, acute illness or other abnormal states to normal, the autopilot voice intervention is performed to ask whether the driver's authority needs to be returned; If the driver's status data returns to normal and the driving accident risk data is lower than the preset intervention threshold during the autonomous driving emergency takeover intervention or multiple emergency takeover intervention states, the autonomous driving voice intervention is performed to prompt the driver to return authority; If the driver's status data returns to normal during the autonomous driving emergency takeover intervention or multiple emergency takeover intervention states, and the driving accident risk data is higher than the preset intervention threshold and lower than the preset danger threshold, autonomous driving voice intervention will be performed to ask whether the driver's authority needs to be returned.

9. The method for determining automatic driving intervention based on multi-point tactile sensing according to claim 6, characterized in that: Mode 3 includes the following optional implementations: 1) Maintaining limited assisted takeover intervention of the automatic driving system; the limited assisted takeover intervention includes switching on and off the indicator lights based on the driving behavior prediction data, specifically: if the driving behavior prediction data is to turn left or right, the left turn signal or right turn signal of the car is turned on accordingly, and turned off at other times; 2) The driver requires the automatic driving system to assist in taking over the intervention according to the needs; the auxiliary takeover intervention includes the automatic driving system judging the driving behavior prediction data, specifically steering fine control and speed fine control; If the driver status data is dull, storing the current moment data to a reserved unit of the multi-point tactile data acquisition device for monitoring; If the driver status data is tense, the current moment data is stored in a reserved unit of the multi-point tactile data acquisition device for monitoring; If the driver's status data is acute illness or other abnormal status, the current moment data is stored in a reserved unit of the multi-point tactile data acquisition device for monitoring; If the driving accident risk data is higher than a preset danger threshold, storing the current moment data to a reserved unit of the multi-point tactile data acquisition device; 3) If the driver's status data is sluggish, the automatic driving system voice intervention will prompt the driver to take a rest; 4) If the driver's status data indicates that the driver is nervous, the autonomous driving system will intervene with voice and provide driving advice for that section of the road; 5) If the driver's status data is acute illness or other abnormal status, the automatic driving system intervenes with voice, asking the driver whether to accept the partial judgment of the emergency takeover intervention of the automatic driving system. If there is no feedback or agreement, the partial judgment of the emergency takeover intervention of the automatic driving system is performed; wherein, the partial judgment of the emergency takeover intervention includes online help and emergency medical contact; 6) If the driver's status data indicates acute illness or other abnormal conditions, or the driving accident risk data is higher than the preset intervention threshold, the autonomous driving system intervenes with voice, asking the driver whether he accepts the full takeover of the autonomous driving system. If there is no feedback or consent, the autonomous driving system will take over the full takeover; the full takeover includes overall control of the vehicle under existing laws and regulations; If the driver's status data is dull, nervous, or acutely ill, or other abnormal conditions, and the autonomous driving system has not been fully taken over and intervened, if the driving accident risk data is higher than the preset danger threshold, the autonomous driving system will be fully taken over and intervened; wherein, the said full takeover and intervention includes the overall control of the car under existing laws and regulations; 7) If the driver's status data is still in a state of dullness, tension, acute illness or other abnormal state within the preset recovery time threshold during the full takeover of the autonomous driving system, the full takeover of the autonomous driving system will continue; wherein, the full takeover includes the overall control of the vehicle under existing laws and regulations; If the driver is in full autopilot mode and the driver's status data changes from dullness, tension, acute illness or other abnormal conditions to normal, the driver will be asked whether the driver's authority needs to be returned. If the driver's status data returns to normal and the driving accident risk data is lower than the preset danger threshold when the autonomous driving fully takes over, the autonomous driving voice intervention will be carried out to ask whether the driver's authority needs to be returned.

10. An automatic driving intervention identification system based on multi-point tactile sensing, characterized in that: The system is implemented by the autonomous driving intervention determination method based on multi-point tactile sensing as described in any one of claims 1 to 9, including: The collecting unit is at least arranged on the surface of the steering wheel and the surface of the pedal, and collects at least the driver's multi-point tactile data of the steering wheel and the pedal; wherein the multi-point tactile data includes kinesthetic and vibration signals; the kinesthetic signals include force, position, and torque; the vibration signals include friction, temperature, and humidity, wherein the force includes normal force and tangential force; A storage unit, used to store multi-point tactile data at a certain moment, and to store and release data in real time as the moment changes; An analysis unit, used to analyze the multi-point tactile data to obtain a driving behavior prediction result, a driver state analysis result, and a driving accident risk prediction result; A reserved unit is used to store multi-point tactile data at specific moments; the specific moments include moments when the accident prediction exceeds a threshold and moments when the driver's state is abnormal.

Images