Man-machine cooperative control system and method based on driver risk cognitive ability

Abstract

The invention discloses a man-machine cooperative control system and method based on the risk cognitive ability of a driver. The method comprises the steps that information is collected based on an information collection assembly; based on the motion state information and the multi-modal information, carrying out driving risk situation modeling so as to obtain the driving risk situation of the vehicle; based on the risk cognition monitoring model, the risk cognition ability of the driver is obtained; performing dynamic distribution of the driving right by using a fuzzy rule to obtain a distribution result of the driving right; a cooperative control strategy is formulated by using an optimization theory so as to control a control assembly of the vehicle and execute a man-machine co-driving collision avoidance operation; according to the method, a driving risk situation model is constructed, physiological data are analyzed in combination with a risk cognition monitoring model, quantitative evaluation of the risk cognition ability of a driver is achieved, risk levels and cognition levels are dynamically fused through fuzzy rules, man-machine driving permissions are allocated, collision avoidance operation is executed according to a cooperative control strategy, and the risk cognition ability of the driver is improved. The defect that a traditional system depends on a single environment risk decision is overcome.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent networked vehicles, in particular to a man-machine collaborative control system and method based on driver risk cognitive ability. BACKGROUND

[0002] According to the J3016 standard formulated by the International Society of Automotive Engineers (SAE International), automatic driving technology is divided into six levels from L0 to L5, and L2 and L3 level vehicles have gradually become the current mainstream. For such vehicles, the driving operation subject is the automatic driving system, and the driver needs to continuously monitor the surrounding environment and take over control when the system requests or in emergency situations.

[0003] However, practice shows that during L2-L3 automatic driving, the driver is prone to distraction, situational awareness degradation, and risk cognitive ability decline due to the monotony of the external environment and the lack of task engagement in the passive monitoring state, significantly reducing the timeliness and reliability of the takeover intervention. Therefore, how to effectively monitor the driver's risk cognitive ability, improve the driver's risk cognitive level, and integrate it into man-machine intelligent collaborative control is one of the keys to ensuring the safety of L2~L3 automatic driving vehicles. In addition, the risk situation in the driving environment is not static, but continuously and dynamically complex in the time and space dimensions, affecting not only the state behavior of the driver, but also the decision-making and control of the automatic driving system. Therefore, how to accurately identify the current driving risk situation and integrate it into the driver's risk cognitive ability monitoring and man-machine intelligent collaborative control is another key to ensuring the safety of L2~L3 automatic driving vehicles. SUMMARY

[0004] The present application aims to overcome the deficiencies in the prior art and provide a man-machine collaborative control system and method based on driver risk cognitive ability to improve the driver's risk cognitive ability and optimize man-machine collaborative control.

[0005] To achieve the above-mentioned purpose, the present application adopts the following technical solutions,

[0006] In a first aspect, the present application provides a man-machine collaborative control system based on driver risk cognitive ability, comprising an information collection component and a cloud subsystem.

[0007] The information collection component comprises:

[0008] A vehicle information collection module for collecting motion state information of the vehicle;

[0009] An environmental information collection module for collecting multi-modal information of surrounding vehicles and road environment;

[0010] The physiological information collection module is configured to collect physiological information of the driver;

[0011] The cloud subsystem comprises:

[0012] The risk situation modeling module is in communication connection with the vehicle information collection module and the environment information collection module, respectively, and is configured to perform driving risk situation modeling according to the motion state information of the ego vehicle, the multi-modal information of the surrounding vehicles and the road environment, so as to obtain the risk situation of the driving of the ego vehicle.

[0013] The risk cognition monitoring module is in communication connection with the physiological information collection module and the risk situation modeling module, respectively, and is configured to input the physiological information of the driver and the risk situation of the driving of the ego vehicle into a pre-trained risk cognition monitoring model, so as to obtain the risk cognition ability of the driver.

[0014] The driving right allocation module is in communication connection with the risk situation modeling module and the risk cognition monitoring module, respectively, and is configured to perform dynamic allocation of driving right by using fuzzy rules according to the risk situation of the driving of the ego vehicle and the risk cognition ability of the driver, so as to obtain an allocation result of the driving right.

[0015] The co-driving control module is in communication connection with the risk situation modeling module, the driving right allocation module and the vehicle information collection module, respectively, and is configured to formulate a cooperative control strategy by using optimization theory according to the risk situation of the driving of the ego vehicle, the allocation result of the driving right and the motion state information of the ego vehicle, and control the steering components of the vehicle based on the cooperative control strategy, so as to perform a collision avoidance operation of human-machine co-driving.

[0016] As an optional implementation manner, the cloud subsystem further comprises a cognition ability improving module.

[0017] The cognition ability improving module is in communication connection with the risk cognition monitoring module, and is configured to retrieve a corresponding risk cognition ability improving scheme from a pre-stored risk cognition ability improving scheme set according to the risk cognition ability of the driver.

[0018] As an optional implementation manner, the cloud subsystem further comprises a multimedia control module.

[0019] The multimedia control module is in communication connection with the cognition ability improving module, and is configured to control the multimedia components of the vehicle based on the risk cognition ability improving scheme, so as to provide risk warning information to the driver.

[0020] As an optional implementation manner, the cloud subsystem further comprises an image information processing module and a radar information processing module.

[0021] The image information processing module input end is in communication connection with the output end of the environment information collection module, the output end of the image information processing module is in communication connection with the input end of the risk situation modeling module, for pre-processing the image information in the multi-modal information, and sending the pre-processed image information to the risk situation modeling module;

[0022] The radar information processing module input end is in communication connection with the output end of the environment information collection module, the output end of the radar information processing module is in communication connection with the input end of the risk situation modeling module, for pre-processing the radar point cloud information in the multi-modal information, and sending the pre-processed radar point cloud information to the risk situation modeling module.

[0023] As an optional implementation manner, the cloud end sub-system further comprises an electroencephalogram information processing module and an eye movement information processing module;

[0024] The input end of the electroencephalogram information processing module is in communication connection with the output end of the physiological information collection module, the output end of the electroencephalogram information processing module is in communication connection with the input end of the risk cognition monitoring module, for pre-processing the electroencephalogram information in the physiological information, and sending the pre-processed electroencephalogram information to the risk cognition monitoring module.

[0025] The input end of the eye movement information processing module is in communication connection with the output end of the physiological information collection module, the output end of the eye movement information processing module is in communication connection with the input end of the risk cognition monitoring module, for pre-processing the eye movement information in the physiological information, and sending the pre-processed eye movement information to the risk cognition monitoring module.

[0026] In a second aspect, the present application provides a man-machine collaborative control method based on driver risk cognitive ability, which is realized based on the man-machine collaborative control system as described above, and comprises the following steps:

[0027] Collecting motion state information of the ego vehicle based on the vehicle information collection module;

[0028] Collecting multi-modal information of surrounding vehicles and road environment based on the environment information collection module;

[0029] The risk situation modeling module acquires the motion state information of the ego vehicle and the multi-modal information, to perform driving risk situation modeling, and obtain the risk situation of the ego vehicle driving;

[0030] Collecting physiological information of the driver based on the physiological information collection module;

[0031] The risk cognition monitoring module acquires the risk situation and the physiological information, and obtains the risk cognitive ability of the driver based on the pre-trained risk cognition monitoring model;

[0032] The driving right allocation module obtains the risk situation and the risk cognitive ability, performs dynamic allocation of the driving right by using fuzzy rules, so as to obtain an allocation result of the driving right;

[0033] The co-driving control module obtains the risk situation, the allocation result and motion state information of the ego vehicle, formulates a cooperative control strategy by using optimization theory, and controls a steering component of the vehicle based on the cooperative control strategy, so as to perform a collision avoidance operation of human-machine co-driving.

[0034] As an optional implementation manner, the method further comprises:

[0035] The cognitive ability improving module obtains the risk cognitive ability of the driver, and based on the risk cognitive ability, retrieves a corresponding risk cognitive ability improving scheme from a pre-stored risk cognitive ability improving scheme set;

[0036] The multimedia control module obtains the risk cognitive ability improving scheme, and based on the risk cognitive ability improving scheme, controls a multimedia component of the vehicle, so as to provide risk warning information to the driver.

[0037] As an optional implementation manner, the obtaining of the motion state information of the ego vehicle and the multi-modal information, and the modeling of the driving risk situation, so as to obtain a driving risk situation of the ego vehicle, specifically comprises the following steps:

[0038] Based on the motion state information of the ego vehicle and the multi-modal information, a plurality of traffic elements in the current driving scene are identified;

[0039] The interaction among the plurality of traffic elements and the interaction coupling mechanism of the interaction on the driving risk are analyzed, so as to construct a macro driving risk scene;

[0040] Based on the macro driving risk scene, data of a preset macro risk index is extracted, and risk rating is performed according to the macro risk index data by using the analytic hierarchy process, so as to obtain a scene risk rating;

[0041] The macro risk index comprises collision severity, number of key risk elements and scene complexity;

[0042] Based on a pre-trained large language model, a micro driving risk process is constructed by using the macro driving risk scene;

[0043] Based on the micro driving risk process, data of a preset micro risk index is extracted, and risk rating is performed according to the micro risk index data by using the analytic hierarchy process, so as to obtain a collision risk rating;

[0044] The micro risk index comprises collision probability, collision time and collision energy;

[0045] The scene risk rating and the collision risk rating are integrated, a predefined fuzzy rule base is used to determine a total driving risk level;

[0046] The scene risk rating, the collision risk rating and the total driving risk level are fused to model a driving risk situation and obtain a risk situation of self-driving.

[0047] As an optional implementation manner, a specific training process of the pre-trained risk cognition monitoring model is as follows:

[0048] An overtake data set containing a plurality of overtake event data is acquired;

[0049] The overtake event data includes overtake performance data of a driver after taking over control in a specific driving risk situation and original physiological response data of the driver within a preset time period before the overtake event occurs;

[0050] The overtake performance data includes an overtake reaction time, an overtake operation characteristic parameter and an actual collision avoidance result;

[0051] According to the overtake performance data, a logistic regression method is used to quantify a risk cognition level of the driver corresponding to each overtake event to obtain a driver risk cognition level value;

[0052] The original physiological response data is labeled according to the driver risk cognition level value as a label;

[0053] An Attention-LSTM network is used to construct a risk cognition monitoring model;

[0054] The risk cognition monitoring model is trained based on the labeled physiological response data and the corresponding label to obtain a trained risk cognition monitoring model.

[0055] As an optional implementation manner, a function expression of the fuzzy rule is as follows:

[0056]

[0057]

[0058] In the formula, f represents a driving right; f represents a driving right initial fixed value; f represents a driving right dynamic adjustment amount; f represents a driving risk situation; f represents a driver risk cognition level; and f () represents a fuzzy rule.

[0059] The embodiments provided by the application have the following beneficial effects: ​

[0060] The present application fuses vehicle motion state, environmental multi-modal information and driver physiological data, constructs a driving risk situation model, analyzes physiological data in combination with a risk cognition monitoring model, realizes quantitative evaluation of driver risk cognition ability, dynamically fuses risk situation grades and cognition ability levels through fuzzy rules, and real-time distributes human-machine driving authority, and then generates a collaborative control strategy according to optimization theory to execute collision avoidance operation, solves the defects of traditional systems relying on single environmental risk decision, and reduces the accident risk caused by driver cognitive delay.

[0061] The present application effectively improves the comprehensiveness and accuracy of driving risk situation evaluation by fusing multi-source information of people, vehicles, roads and environment, and adopting a three-level hierarchical modeling method of constructing macro risk scene, micro risk process and overall risk level, so that the driver and the system can obtain more comprehensive and accurate driving risk understanding. BRIEF DESCRIPTION OF DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other embodiments can also be obtained by those skilled in the art according to these drawings.

[0063] Figure 1 The structure block diagram of the human-machine collaborative control system in the embodiments of the present application is shown;

[0064] Figure 2 The layout diagram of the human-machine intelligent collaborative control system in the embodiments of the present application is shown;

[0065] Figure 3 The flowchart of the human-machine collaborative control method in the embodiments of the present application is shown;

[0066] Figure 4 The flowchart of the driving risk situation modeling in the embodiments of the present application is shown;

[0067] Figure 5 The flowchart of the driver risk cognition monitoring in the embodiments of the present application is shown;

[0068] Figure 6 The flowchart of the human-machine collaborative control in the embodiments of the present application is shown. DETAILED DESCRIPTION

[0069] The features and exemplary embodiments of the various aspects of the present application will be described in detail below. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one of ordinary skill in the art that the present application can be practiced without some or all of these specific details. The description of the embodiments is merely intended to provide a better understanding of the present application by showing examples of the present application.

[0070] In the description of the embodiments of the present application, it should be noted that, unless otherwise explicitly specified and limited, the terms "communication" and "communication connection" should be understood broadly, i.e., it means that both parties of the communication have a link relationship capable of transmitting data and other information content, for example, it can be a fixed connection through a data line, or a detachable connection or an integral connection of a data connection line; it can be a connection through an electric wire, or other forms of electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium. For those of ordinary skill in the art, the specific meaning of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0071] The present application will be described in further detail below in conjunction with the accompanying drawings and specific embodiments.

[0072] Embodiment 1

[0073] As shown in the figure, the present embodiment provides a man-machine cooperative control system based on driver risk perception ability, which comprises an information acquisition component and a cloud subsystem; Figures 1-2

[0074] The information acquisition component comprises a vehicle information acquisition module, an environment information acquisition module and a physiological information acquisition module; the cloud subsystem comprises a risk situation modeling module, a risk perception monitoring module, a driving right allocation module and a co-driving control module.

[0075] For example, the output end of the vehicle information acquisition module is in communication connection with the input end of the risk situation modeling module and the co-driving control module respectively, for acquiring the motion state information of the ego vehicle.

[0076] The motion state information of the ego vehicle includes but is not limited to the position information of the ego vehicle, the speed information of the ego vehicle and the acceleration information of the ego vehicle.

[0077] The output end of the environment information acquisition module is in communication connection with the input end of the risk situation modeling module, for acquiring the multi-modal information of surrounding vehicles and road environment.

[0078] The multi-modal information of surrounding vehicles and road environment includes but is not limited to image information and radar point cloud information.

[0079] The image information includes but is not limited to road environment information, signal light information, obstacle information and weather information.​

[0080] The radar point cloud information includes, but is not limited to, position information, speed information, and acceleration information of surrounding other vehicles:

[0081] The output end of the physiological information acquisition module is in communication connection with the input end of the risk cognition monitoring module, and is configured to acquire physiological information of the driver;

[0082] The physiological information of the driver includes, but is not limited to, electroencephalogram information of the driver and eye movement information of the driver;

[0083] The output end of the risk situation modeling module is in communication connection with the input end of the risk cognition monitoring module, and is configured to perform driving risk situation modeling according to the motion state information of the ego vehicle and the multi-modal information of surrounding vehicles and road environment, so as to obtain a driving risk situation of the ego vehicle;

[0084] The output end of the risk cognition monitoring module is in communication connection with the input end of the driving right allocation module, and is configured to input the physiological information of the driver and the driving risk situation of the ego vehicle into a pre-trained risk cognition monitoring model, so as to obtain a risk cognition ability of the driver;

[0085] The input end of the driving right allocation module is in communication connection with the output end of the risk situation modeling module, and is configured to perform dynamic allocation of driving right according to the driving risk situation of the ego vehicle and the risk cognition ability of the driver by using fuzzy rules, so as to obtain an allocation result of the driving right;

[0086] The input end of the co-driving control module is in communication connection with the output ends of the risk situation modeling module and the driving right allocation module, respectively, and is configured to formulate a cooperative control strategy by using optimization theory according to the driving risk situation of the ego vehicle, the allocation result of the driving right, and the motion state information of the ego vehicle, and control a steering component of the vehicle based on the cooperative control strategy to perform a collision avoidance operation of man-machine co-driving.

[0087] Illustratively, the cloud subsystem further includes a cognition ability improving module and a multimedia control module;

[0088] Specifically, the cognition ability improving module is in communication connection with the risk cognition monitoring module, and is configured to retrieve a corresponding risk cognition ability improving scheme from a pre-stored risk cognition ability improving scheme set according to the risk cognition ability of the driver;

[0089] The multimedia control module is in communication connection with the cognition ability improving module, and is configured to control a multimedia component of the vehicle based on the risk cognition ability improving scheme to provide risk warning information to the driver.

[0090] In some embodiments, the cloud subsystem further includes an image information processing module, a radar information processing module, an electroencephalogram information processing module, and an eye movement information processing module;

[0091] Specifically, the image information processing module input end is in communication connection with the output end of the environment information collection module, the output end of the image information processing module is in communication connection with the input end of the risk situation modeling module, for pre-processing the image information in the multi-modal information, and sending the pre-processed image information to the risk situation modeling module;

[0092] The radar information processing module input end is in communication connection with the output end of the environment information collection module, and the output end of the radar information processing module is in communication connection with the input end of the risk situation modeling module, for pre-processing the radar point cloud information in the multi-modal information, and sending the pre-processed radar point cloud information to the risk situation modeling module.

[0093] The input end of the electroencephalogram information processing module is in communication connection with the output end of the physiological information collection module, and the output end of the electroencephalogram information processing module is in communication connection with the input end of the risk cognition monitoring module, for pre-processing the electroencephalogram information in the physiological information, and sending the pre-processed electroencephalogram information to the risk cognition monitoring module.

[0094] The input end of the eye movement information processing module is in communication connection with the output end of the physiological information collection module, and the output end of the eye movement information processing module is in communication connection with the input end of the risk cognition monitoring module, for pre-processing the eye movement information in the physiological information, and sending the pre-processed eye movement information to the risk cognition monitoring module.

[0095] In some embodiments, the steering assembly of the vehicle includes but is not limited to a vehicle steering mechanism, a vehicle acceleration mechanism, and a vehicle braking mechanism;

[0096] In some embodiments, the multimedia assembly includes but is not limited to a voice reminding unit, a light reminding unit, and a display reminding unit; wherein the voice reminding unit can be a vehicle-mounted player; the light reminding unit can be a vehicle-mounted atmosphere lamp; and the display reminding unit can be a vehicle-mounted display;

[0097] In some embodiments, the vehicle information collection module includes but is not limited to a GPS, a speed sensor, and an acceleration sensor;

[0098] The environment information collection module includes but is not limited to a plurality of high-definition cameras installed on the outside of the vehicle body, a 192-line laser radar, an ultrasonic radar, and a millimeter wave radar;

[0099] The physiological information collection module includes but is not limited to a non-embedded electroencephalogram collection device and a wearable eye movement collection device;

[0100] Embodiment 2

[0101] As Figure 3As shown, the embodiment provides a man-machine collaborative control method based on driver risk perception ability, which is realized based on the embodiment 1 as described above, and includes the following steps:

[0102] S1, acquiring motion state information of the ego vehicle based on a vehicle information acquisition module;

[0103] S2, acquiring multi-modal information of surrounding vehicles and road environment based on an environment information acquisition module;

[0104] S3, a risk situation modeling module acquires the motion state information of the ego vehicle and the multi-modal information to model the driving risk situation, and obtains the risk situation of the ego vehicle driving;

[0105] As Figure 4 shown, in some embodiments, the acquisition of the motion state information of the ego vehicle and the multi-modal information in step S3 to model the driving risk situation to obtain the risk situation of the ego vehicle driving specifically includes the following steps:

[0106] S301, based on the motion state information of the ego vehicle and the multi-modal information, a plurality of traffic elements in the current driving scene are identified;

[0107] wherein the traffic elements include but are not limited to pedestrians, vehicles, roads and surrounding environment;

[0108] S302, the interaction between the plurality of traffic elements and the interaction coupling mechanism of the interaction on driving risk are analyzed to construct a macro driving risk scene;

[0109] S303, based on the macro driving risk scene, data of preset macro risk indicators are extracted, and risk rating is performed according to the macro risk indicator data through analytic hierarchy process to obtain scene risk rating;

[0110] The macro risk indicators include collision severity, number of key risk elements and scene complexity;

[0111] S304, based on a pre-trained large language model, a micro driving risk process is constructed using the macro driving risk scene;

[0112] Specifically, the pre-trained large language model in step S304 has a function expression as follows:

[0113]

[0114] wherein, is the historical motion sequence of the ego vehicle; is the historical motion sequence of the i-th background entity; is a macroscopic driving risk scene; N is the length of the historical motion sequence; T is the number of surrounding background entities; is the future motion sequence of the i+1th background entity; M is the length of the future motion sequence; is an encoder; is a decoder; is a high-order feature vector;

[0115] The above-mentioned background entity refers to an objective thing or object that can be perceived, recognized and has independent existence meaning in a specific scene, such as a vehicle, a pedestrian, a building, etc.

[0116] S305, based on the microscopic driving risk process, extracting data of a preset microscopic risk indicator, and performing risk rating on the microscopic risk indicator data through an analytic hierarchy process to obtain a collision risk rating;

[0117] The microscopic risk indicator includes a collision probability, a collision time and a collision energy;

[0118] S306, comprehensively rating the scene risk rating and the collision risk rating, and determining a driving overall risk level by using a predefined fuzzy rule base;

[0119] The predefined fuzzy rule base is shown in Table 1:

[0120]

[0121] Table 1 is a driver takeover ability evaluation table;

[0122] The scene risk rating is a current scene risk level, and the current scene risk level includes safety SA, low risk LR, medium risk MR, medium-high risk MHR and high risk HR;

[0123] S307, fusing the scene risk rating, the collision risk rating and the driving overall risk level to model a driving risk situation and obtain a risk situation of a vehicle.

[0124] In this embodiment, by fusing multi-source information of people, vehicles, roads and environments, and adopting a three-level hierarchical modeling method of constructing a macroscopic risk scene, a microscopic risk process and an overall risk level, the scene risk indicator and the collision risk indicator are quantitatively rated by an analytic hierarchy process to generate a scene risk rating and a collision risk rating. Then, the scene risk rating and the collision risk rating are fused by using fuzzy rules to calculate a driving overall risk level. Finally, by integrating the scene risk rating, the collision risk rating and the driving overall risk level, a driving risk situation is modeled by using digital technology.

[0125] The comprehensive and accurate driving risk situation assessment is effectively improved, so that the driver and the system can obtain more comprehensive and accurate driving risk understanding.

[0126] S4, collecting physiological information of the driver based on a physiological information collection module;

[0127] S5, obtaining the risk situation and the physiological information by a risk cognition monitoring module, and obtaining the risk cognition ability of the driver based on a pre-trained risk cognition monitoring model;

[0128] As shown in Figure 5 In some embodiments, the pre-trained risk cognition monitoring model in step S5 has the following specific training process:

[0129] S501, obtaining a takeover data set containing a plurality of takeover event data;

[0130] The takeover event data includes takeover performance data of the driver after taking over control in a specific driving risk scenario and original physiological response data of the driver within a preset time period before the takeover event occurs;

[0131] The takeover performance data includes takeover reaction time, takeover operation characteristic parameters, and actual collision avoidance results;

[0132] In this embodiment, the takeover operation characteristic parameters include but are not limited to maximum steering wheel angular velocity, average steering wheel angular velocity, maximum deceleration, average deceleration, and minimum distance between the ego vehicle and the obstacle; and the collision avoidance result is successful collision avoidance or collision occurrence;

[0133] S502, according to the takeover performance data, quantifying the risk cognition level of the driver corresponding to each takeover event by using a logistic regression method to obtain a driver risk cognition level value;

[0134] Wherein, the driver risk cognition level value is divided into high risk cognition level, general risk cognition level and low risk cognition level;

[0135] S503, according to the driver risk cognition level value as a label, labeling the corresponding original physiological response data;

[0136] S504, constructing a risk cognition monitoring model based on an Attention-LSTM network;

[0137] The Attention-LSTM network is an improved model that adds an attention mechanism to an existing LSTM (Long Short-Term Memory), so that the model can dynamically identify key feature nodes in the physiological data stream, and through an adaptive weighting strategy, the physiological response fragments that have a significant indication on the risk cognitive state are strengthened, thereby improving the pertinence of feature extraction, effectively analyzing the mapping relationship between the driver's physiological information and the real-time driving risk situation, establishing a quantitative evaluation capability of the driver's risk cognitive level, and solving the problem of insufficient sensitivity of the traditional model to important features in long time series data.

[0138] S505, training a risk cognitive monitoring model based on the labeled physiological response data and the corresponding labels to obtain a trained risk cognitive monitoring model.

[0139] In this embodiment, during the training of the risk cognitive monitoring model, first, based on the takeover reaction time and takeover operation characteristic parameters and other external behavior performances of the driver in the takeover event, the risk cognitive level of the driver is quantitatively evaluated through a logistic regression algorithm, to realize the inference from observable takeover performance data to the internal risk cognitive state; then, based on the quantitatively obtained risk cognitive level value, the original physiological response data in a preset time period before the corresponding takeover event occurs is labeled, thereby establishing the association mapping between the takeover behavior result and the pre-event physiological features. Finally, using the labeled physiological response data set, the risk cognitive monitoring model is trained, so that the model can identify the key patterns representing cognitive ability from the real-time collected physiological signals.

[0140] S6, the risk situation and the risk cognitive ability are obtained by the driving right allocation module, and fuzzy rules are used for dynamic allocation of driving right, to obtain a driving right allocation result, as shown in Figure 6 .

[0141] In some embodiments, the function expression of the fuzzy rule in step S6 is:

[0142]

[0143]

[0144] In the formula, represents the driving right; represents the initial fixed value of the driving right; represents the dynamic adjustment amount of the driving right; represents the driving risk situation; represents the risk cognitive level of the driver; and

[0145] S7, the co-driving control module obtains the risk situation, the allocation result and the motion state information of the ego vehicle, formulates a cooperative control strategy by using optimization theory, and controls the steering components of the vehicle based on the cooperative control strategy to perform the collision avoidance operation of human-machine co-driving.

[0146] In some embodiments, the step of formulating a cooperative control strategy by using optimization theory in step S7 specifically includes the following steps:

[0147] determining an expected driving path of the driver as the leader based on the allocation result of the driving right, the motion state information of the ego vehicle, the risk situation and the steering input signal of the driver;

[0148] the intelligent driving system as the follower aims to minimize the degree of deviation of the control behavior of the driver and the intelligent driving system from their respective expected paths; the expected path of the intelligent driving system is consistent or cooperative with the expected path of the driver under the constraint of the allocation result of the driving right;

[0149] based on the master-slave game theory framework, the control decision Nash equilibrium point of the driver and the intelligent driving system is solved by using the reverse induction method to obtain the optimal control decision;

[0150] the intelligent driving system outputs control instructions to the steering components of the vehicle according to the optimal control decision.

[0151] In some embodiments, the method further includes:

[0152] S8, the cognitive ability improvement module obtains the risk cognitive ability of the driver, and based on the risk cognitive ability, retrieves a corresponding risk cognitive ability improvement scheme from a pre-stored risk cognitive ability improvement scheme set;

[0153] S9, the multimedia control module obtains the risk cognitive ability improvement scheme, and controls the multimedia components of the vehicle based on the risk cognitive ability improvement scheme to provide risk warning information to the driver.

[0154] In this embodiment, the cognitive ability improvement module matches and calls the adaptive improvement strategy from the pre-constructed risk cognitive ability improvement scheme set according to the real-time evaluated risk cognitive level value of the driver, and the multimedia control module responds to the improvement strategy and dynamically generates warning information matched with the current driving risk level through the multimedia output interfaces such as the interactive interface animation of the vehicle-mounted player, vehicle-mounted atmosphere lamp and vehicle-mounted display in the vehicle, thereby strengthening the perception and understanding ability of the driver to potential risks.

[0155] To sum up, the embodiment fuses the vehicle motion state, the environment multi-modal information and the driver physiological data, constructs a driving risk situation model, analyzes the physiological data in combination with a risk cognition monitoring model, realizes quantitative evaluation of the driver risk cognition ability, dynamically fuses the risk situation grade and the cognition ability level through fuzzy rules, allocates the human-machine driving right in real time, generates a cooperative control strategy according to optimization theory to execute an avoidance operation, solves the defects of the traditional system depending on a single environment risk decision, and reduces the accident risk caused by the driver cognition delay.

[0156] The embodiment dynamically allocates the driving right through fuzzy rules based on the driving risk situation and the monitoring result of the driver risk cognition ability: when the driver is in a low risk cognition level, the driving right is transferred to the co-driving control module to ensure driving safety; when the driver is in a general or high risk cognition level, the control right of the driver is ensured to guarantee the driving experience, thereby effectively optimizing the environment adaptability and user acceptability of the human-machine cooperative control, improving the driving safety while taking into account the driving experience.

[0157] The above is only the preferred embodiment of the present application, and it is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be realized in other specific forms without departing from the spirit or basic characteristics of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, the scope of the present application is defined by the appended claims rather than the above description, and all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application.

Claims

1. A human-machine collaborative control system based on the driver's risk perception ability, characterized in that, Includes information collection components and a cloud subsystem; The information acquisition component includes: The vehicle information collection module is used to collect the vehicle's motion status information; The environmental information acquisition module is used to collect multimodal information about the surrounding vehicles and road environment; The physiological information collection module is used to collect the driver's physiological information; The cloud subsystem includes: The risk situation modeling module is connected to the vehicle information acquisition module and the environmental information acquisition module respectively. It is used to perform driving risk situation modeling based on the vehicle's motion state information, the multimodal information of surrounding vehicles and road environment, so as to obtain the risk situation of the vehicle's driving. The risk perception monitoring module is connected to the physiological information acquisition module and the risk situation modeling module respectively. It is used to input the driver's physiological information and the risk situation of the vehicle into the pre-trained risk perception monitoring model to obtain the driver's risk perception ability. The driving rights allocation module is connected to the risk situation modeling module and the risk perception monitoring module respectively. It is used to dynamically allocate driving rights based on the risk situation of the vehicle and the driver's risk perception ability using fuzzy rules to obtain the driving rights allocation result. The co-driving control module is connected to the risk situation modeling module, the driving rights allocation module, and the vehicle information acquisition module. It is used to formulate a cooperative control strategy based on the risk situation of the vehicle, the driving rights allocation result, and the motion state information of the vehicle, and to control the vehicle's control components based on the cooperative control strategy to perform collision avoidance operations in human-machine co-driving.

2. The system according to claim 1, characterized in that, The cloud-based subsystem also includes a cognitive ability enhancement module; The cognitive ability enhancement module is communicatively connected to the risk cognition monitoring module and is used to retrieve the corresponding risk cognition ability enhancement scheme from the pre-stored risk cognition ability enhancement scheme set according to the driver's risk cognition ability.

3. The system according to claim 2, characterized in that, The cloud subsystem also includes a multimedia control module; The multimedia control module is communicatively connected to the cognitive ability enhancement module and is used to control the vehicle's multimedia components based on the risk cognitive ability enhancement scheme in order to provide risk warning information to the driver.

4. The system according to claim 1, characterized in that, The cloud-based subsystem also includes an image information processing module and a radar information processing module; The input end of the image information processing module is communicatively connected to the output end of the environmental information acquisition module, and the output end of the image information processing module is communicatively connected to the input end of the risk situation modeling module. It is used to preprocess the image information in the multimodal information and send the preprocessed image information to the risk situation modeling module. The input end of the radar information processing module is communicatively connected to the output end of the environmental information acquisition module, and the output end of the radar information processing module is communicatively connected to the input end of the risk situation modeling module. It is used to preprocess the radar point cloud information in the multimodal information and send the preprocessed radar point cloud information to the risk situation modeling module.

5. The system according to claim 1, characterized in that, The cloud-based subsystem also includes an EEG information processing module and an eye-tracking information processing module; The input end of the EEG information processing module is communicatively connected to the output end of the physiological information acquisition module, and the output end of the EEG information processing module is communicatively connected to the input end of the risk cognition monitoring module. It is used to preprocess the EEG information in the physiological information and send the preprocessed EEG information to the risk cognition monitoring module. The input end of the eye movement information processing module is communicatively connected to the output end of the physiological information acquisition module, and the output end of the eye movement information processing module is communicatively connected to the input end of the risk perception monitoring module. It is used to preprocess the eye movement information in the physiological information and send the preprocessed eye movement information to the risk perception monitoring module.

6. A human-machine collaborative control method based on the driver's risk perception ability, characterized in that, Based on the human-machine collaborative control system as described in any one of claims 1 to 5, the system includes the following steps: The vehicle's motion status information is collected based on the vehicle information collection module; The environmental information acquisition module collects multimodal information about the surrounding vehicles and road environment; The risk situation modeling module acquires the vehicle's motion state information and the multimodal information to perform driving risk situation modeling and obtain the risk situation of the vehicle's driving. The physiological information of the driver is collected based on the physiological information collection module; The risk perception monitoring module acquires the risk situation and the physiological information, and obtains the driver's risk perception ability based on the pre-trained risk perception monitoring model; The driving rights allocation module acquires the risk situation and the risk perception ability, and uses fuzzy rules to dynamically allocate driving rights in order to obtain the driving rights allocation result; The co-driving control module acquires the risk situation, the allocation result, and the vehicle's motion state information, formulates a cooperative control strategy using optimization theory, and controls the vehicle's control components based on the cooperative control strategy to perform collision avoidance operations in human-machine co-driving.

7. The method according to claim 6, characterized in that, Also includes: The cognitive ability enhancement module acquires the driver's risk cognitive ability and, based on the risk cognitive ability, retrieves the corresponding risk cognitive ability enhancement scheme from the pre-stored risk cognitive ability enhancement scheme set; The multimedia control module acquires a risk awareness enhancement plan and controls the vehicle's multimedia components based on the plan to provide risk warning information to the driver.

8. The method according to claim 6, characterized in that, The process of acquiring the vehicle's motion state information and the multimodal information, and performing driving risk situation modeling to obtain the vehicle's driving risk situation, specifically includes the following steps: Based on the vehicle's motion state information and the multimodal information, various traffic elements in the current driving scenario can be identified; The interactions between the various traffic elements and the interactive coupling mechanism of these interactions on driving risks are analyzed to construct a macro-level driving risk scenario. Based on the aforementioned macro-level driving risk scenario, data of preset macro-level risk indicators are extracted, and risk rating is performed based on the macro-level risk indicator data using the analytic hierarchy process to obtain a scenario risk rating. The macro risk indicators include collision severity, number of key risk factors, and scenario complexity; Based on a pre-trained large language model, a micro-level driving risk process is constructed using the aforementioned macro-level driving risk scenarios. Based on the aforementioned micro-level driving risk process, data of preset micro-level risk indicators are extracted, and risk rating is performed using the analytic hierarchy process based on the micro-level risk indicator data to obtain a collision risk rating. The micro-risk indicators include collision probability, collision time, and collision energy. By combining the scenario risk rating and the collision risk rating, and using a predefined fuzzy rule base, the overall driving risk level is determined. By integrating the scenario risk rating, the collision risk rating, and the overall driving risk level, a driving risk situation is modeled, and the driving risk situation of the vehicle is obtained.

9. The method according to claim 6, characterized in that, The specific training process of the pre-trained risk perception monitoring model is as follows: Retrieve the takeover dataset containing data from multiple takeover events; The takeover event data includes the driver's takeover performance data after taking over control in a specific driving risk scenario and the driver's original physiological response data within a preset time period before the takeover event occurs. The takeover performance data includes takeover reaction time, takeover operation characteristic parameters, and actual collision avoidance results; Based on the takeover performance data, the driver's risk perception level corresponding to each takeover event is quantified using logistic regression to obtain the driver's risk perception level value. The original physiological response data are labeled based on the driver's risk perception level value; A risk perception monitoring model is constructed based on the Attention-LSTM network; The risk perception monitoring model is trained based on the labeled physiological response data and corresponding labels to obtain the trained risk perception monitoring model.

10. The method according to claim 6, characterized in that, The functional expression of the fuzzy rule is: ； ； In the formula, Represents driving rights; This represents the initial fixed value of driving rights; Represents the dynamic adjustment amount of driving rights; Represents the driving risk situation; f represents the driver's risk perception level; f() represents a fuzzy rule.

Images