Driving decision-making method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a driving decision-making method and device, computer equipment, a readable storage medium and a program product. The method comprises the steps of determining a first driving action decision result and a second driving action decision result based on environment information and a preset decision model; determining risk perception parameters based on a preset risk perception model and the physiological signal features; and determining a driving decision based on the risk perception parameter, the first driving action decision result and the second driving action decision result. By adopting the method, the automatic driving decision can be matched with the actual scene of the vehicle, so that the safety of automatic driving is ensured.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a driving decision-making method, apparatus, computer equipment, readable storage medium, and program product. Background Art

[0002] With the development of autonomous driving technology, autonomous driving vehicles are gradually becoming practical. Achieving safe decision-making in autonomous driving is one of the key technologies for autonomous driving vehicles. Traditional autonomous driving technologies often use guided reinforcement learning methods, which only focus on reinforcement learning decisions based on vehicle information. Limited by algorithms and vehicle information, the output results of the vehicle's autonomous driving decisions do not match the safety decision-making requirements based on human factors. Summary of the invention

[0003] Based on this, it is necessary to provide a driving decision-making method, device, computer equipment, readable storage medium and program product to address the above-mentioned technical problems, which can match autonomous driving decisions with human-based safety decisions to ensure the safety of autonomous driving.

[0004] In a first aspect, the present application provides a driving decision-making method, comprising:

[0005] Determining a first driving action decision result and a second driving action decision result based on the environmental information and a preset decision model;

[0006] Determine risk perception parameters based on the preset risk perception model and physiological signal characteristics;

[0007] A driving decision is determined based on the risk perception parameter, the first driving action decision result and the second driving action decision result.

[0008] In one embodiment, determining the driving decision based on the risk perception parameter, the first driving action decision result and the second driving action decision result includes:

[0009] If the risk perception parameter is a first parameter, determining the second driving action decision result as the driving decision, and the first parameter indicates that the vehicle is in a high-risk state;

[0010] If the risk perception parameter is a second parameter, the first driving action decision result is determined as the driving decision, and the second parameter indicates that the vehicle is in a low-risk state.

[0011] In one embodiment, the preset decision model includes a first preset decision model and a second preset decision model, the first preset decision model is obtained based on the training of the reinforcement learning model, the second preset decision model includes one of a vehicle lateral control model, a vehicle longitudinal control model and a vehicle lateral and longitudinal integrated control model, and the first driving action decision result and the second driving action decision result are determined based on the environmental information and the preset decision model, including:

[0012] Obtaining a first driving action decision result based on the environmental information and the first preset decision model;

[0013] Based on the environmental information and the second preset decision model, a second driving action decision result is obtained. In one embodiment, the method further includes:

[0014] Calculating difference data between the driving decision and the first driving action decision result;

[0015] Determining a target optimization parameter of the first preset decision model based on the difference data;

[0016] The first preset decision model is updated based on the target optimization parameter to obtain an updated first preset decision model.

[0017] In one embodiment, determining the target optimization parameter of the first preset decision model based on the difference data includes:

[0018] Calculating a product value of a risk perception parameter and the difference data, and determining the product value as decision difference data between the first driving action decision result and the driving decision;

[0019] Calculating a state action value function for adopting the driving decision under the environmental information;

[0020] Determining an objective function based on the state-action-value function and the decision difference data;

[0021] The parameter gradient of the first preset decision model is updated according to the objective function and the data information in the experience replay pool to obtain the target optimization parameters of the first preset decision model.

[0022] In one embodiment, the method further comprises:

[0023] Collect sample physiological signals under preset dangerous scenarios;

[0024] Extracting features of the sample physiological signal to obtain features of the sample physiological signal;

[0025] Based on the sample physiological signal characteristics and human subjective risk labels, training is performed to obtain a preset risk perception model, where the preset risk perception model characterizes the mapping relationship between the physiological signal characteristics and the human subjective risk labels.

[0026] In a second aspect, the present application also provides a driving decision-making device, comprising:

[0027] A first determination module, used to determine a first driving action decision result and a second driving action decision result based on environmental information and a preset decision model;

[0028] A second determination module is used to determine risk perception parameters based on a preset risk perception model and physiological signal characteristics;

[0029] The third determination module is used to determine the driving decision based on the risk perception parameter, the first driving action decision result and the second driving action decision result.

[0030] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0031] Determining a first driving action decision result and a second driving action decision result based on the environmental information and a preset decision model;

[0032] Determine risk perception parameters based on the preset risk perception model and physiological signal characteristics;

[0033] A driving decision is determined based on the risk perception parameter, the first driving action decision result and the second driving action decision result.

[0034] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0035] Determining a first driving action decision result and a second driving action decision result based on the environmental information and a preset decision model;

[0036] Determine risk perception parameters based on the preset risk perception model and physiological signal characteristics;

[0037] A driving decision is determined based on the risk perception parameter, the first driving action decision result and the second driving action decision result.

[0038] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0039] Determining a first driving action decision result and a second driving action decision result based on the environmental information and a preset decision model;

[0040] Determine risk perception parameters based on the preset risk perception model and physiological signal characteristics;

[0041] A driving decision is determined based on the risk perception parameter, the first driving action decision result and the second driving action decision result.

[0042] The above-mentioned driving decision method, device, computer equipment, readable storage medium and program product determine the first driving action decision result and the second driving action decision result through environmental information, determine the risk perception parameter based on the physiological signal characteristics, and use the risk perception parameter as a judgment condition to determine the first driving action decision result or the second driving action decision result as the driving decision, which can match the decision result of the autonomous driving vehicle with the actual environment of the vehicle, so as to improve the accuracy of the driving decision and further ensure the safety of personnel and vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 is a flowchart of a driving decision-making method in one embodiment;

[0045] Figure 2 is a schematic diagram of a driving decision-making method in one embodiment;

[0046] Figure 3 is a structural block diagram of a driving decision-making device in one embodiment;

[0047] Figure 4 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] In an exemplary embodiment, Figure 1As shown, a driving decision method is provided. This embodiment uses the method applied to a terminal as an example for illustration. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0050] Step 101: Determine a first driving action decision result and a second driving action decision result based on environmental information and a preset decision model.

[0051] The environmental information may represent the information of the environment in which the vehicle is located, for example, the environmental information includes but is not limited to one or more of environmental image information, point cloud information, the vehicle position, the vehicle speed, and the position and speed of surrounding obstacles. The first driving action decision result and the second driving action decision result may represent the behavior that the vehicle should take in combination with the current environmental information.

[0052] Specifically, the terminal can receive environmental information collected by the sensor, and can receive environmental information obtained by processing the collected initial environmental data by the road-side equipment or the vehicle algorithm. The terminal can input the environmental information into a preset decision model and output the first driving action decision result and the second driving action decision result.

[0053] Exemplarily, the terminal can obtain environmental image information through visual sensors, obtain point cloud information by activating radar or millimeter-wave radar, and process the collected initial environmental data such as running time and running distance through road-side equipment or self-vehicle algorithms to obtain information such as the position and speed of the self-vehicle and the position and speed of surrounding obstacles.

[0054] Step 102: Determine risk perception parameters based on a preset risk perception model and physiological signal characteristics.

[0055] Among them, the physiological signal feature can reflect the physiological response of the occupant to the change of the external environment, and can be obtained by extracting the physiological signal. The risk perception parameter can reflect whether the occupant perceives that the vehicle is currently in a risky state.

[0056] Specifically, the terminal can obtain physiological signals collected by the sensor device. The terminal can preprocess the physiological signals to obtain preprocessed physiological signals, and extract features from the preprocessed physiological signals to obtain physiological signal features. The terminal can input the physiological signal features into a preset risk perception model and output risk perception parameters.

[0057] The specific implementation process of the terminal preprocessing the physiological signal may include: the terminal may filter the physiological signal using a filter to obtain a filtered physiological signal; the terminal may collect the physiological signal in a resting state, and use the physiological signal in a resting state to perform baseline correction on the filtered physiological signal to obtain a baseline-corrected physiological signal. The terminal may use a motion artifact removal algorithm to remove motion artifacts from the baseline-corrected physiological signal to obtain a preprocessed physiological signal. The motion artifact removal algorithm may be an algorithm such as a temporal derivative distribution repair (TDDR) algorithm.

[0058] The specific implementation process of the terminal extracting features of the preprocessed physiological signal may include: the terminal may use a time domain extraction tool to extract the time domain features of the preprocessed physiological signal; use a Fourier transform algorithm to convert the time domain features into frequency domain features; extract the time series features of the preprocessed physiological signal based on a deep learning network, and determine the time domain features, frequency domain features and time series features as physiological signal features. In addition, the physiological signal features may be subjected to dimensionality reduction processing.

[0059] The specific implementation process of the terminal inputting physiological signal characteristics into a preset risk perception model to output risk perception parameters may include: inputting physiological signal characteristics into a machine learning model, using machine learning methods to classify them into different risk levels, and determining the risk level as a risk perception parameter.

[0060] Exemplarily, the terminal can collect the blood oxygen signal of the occupant through a blood oxygen detection device and determine the blood oxygen signal as a physiological signal. The blood oxygen detection device can be functional near-infrared spectroscopy (fNIRS). In addition, the terminal can also collect EEG signals from an EEG device on an EEG cap and determine the EEG signals as physiological signals, etc. It should be understood that this is only for illustration and does not specifically limit physiological signals.

[0061] Step 103: Determine a driving decision based on the risk perception parameter, the first driving action decision result, and the second driving action decision result.

[0062] Specifically, the terminal may determine the first driving action decision result as the driving decision, or determine the second driving action decision result as the driving decision, based on the risk perception parameter.

[0063] The above-mentioned driving decision method determines the first driving action decision result and the second driving action decision result through environmental information, determines the risk perception parameter based on the physiological signal characteristics, and uses the risk perception parameter as a judgment condition to determine the first driving action decision result or the second driving action decision result as the driving decision. The decision result of the automatic driving vehicle can be matched with the actual environment of the vehicle to improve the accuracy of the driving decision, which can further ensure the safety of personnel and vehicles.

[0064] In an exemplary embodiment, the specific implementation process of step 103 of “determining a driving decision based on the risk perception parameter, the first driving action decision result and the second driving action decision result” may include:

[0065] If the risk perception parameter is the first parameter, the second driving action decision result is determined as the driving decision; if the risk perception parameter is the second parameter, the first driving action decision result is determined as the driving decision.

[0066] Among them, the first parameter can reflect that the vehicle is in a high-risk state, and the second parameter can reflect that the vehicle is in a low-risk state. The high-risk state can represent that the current state of the vehicle may cause harm to the vehicle and the personnel in the vehicle. The low-risk state can represent that the current state of the vehicle will not cause harm to the vehicle and the personnel in the vehicle. The second driving action decision result can be a conservative driving action decision result. The first driving action decision result can be a driving action decision result output by the driving decision model to be trained.

[0067] Specifically, if the risk perception parameter is the first parameter, the terminal determines that the vehicle is currently in a high-risk state, and determines the second driving action decision result as the driving decision. If the risk perception parameter is the second parameter, the terminal determines that the vehicle is in a normal state or a low-risk state, and determines the first driving action decision result as the driving decision. The terminal can control the driving state of the vehicle based on the driving decision.

[0068] Exemplarily, the first parameter may be 1, and the second parameter may be 0. When the risk perception parameter λ=1, that is, the occupant perceives that the vehicle is in a high-risk state, the second driving action decision result is determined as the driving decision. When the risk perception parameter λ=1, that is, the occupant perceives that the vehicle is in a normal state or a low-risk state, the first driving action decision result is determined as the driving decision. The specific formula for the driving decision may be:

[0069]

[0070] Among them, a is the driving decision; λ is the risk perception parameter; a RL is the first driving action decision result; a IDM It is the second driving action decision result.

[0071] In this embodiment, by using physiological signal characteristics as the basis for judging driving risks, autonomous driving is made safer and more humane. It can monitor the first driving action decision result in real time. When the output of the first driving action decision result does not conform to the actual scenario, it can switch to the second driving action decision result in time to help reinforce the exploration of safe driving strategies through learning.

[0072] In an exemplary embodiment, the specific implementation process of step 101 “determining the first driving action decision result and the second driving action decision result based on the environmental information and the preset decision model” may include:

[0073] Based on the environmental information and the first preset decision model, a first driving action decision result is obtained; based on the environmental information and the second preset decision model, a second driving action decision result is obtained. Wherein, the preset decision model includes a first preset decision model and a second preset decision model, the first preset strategy model can be based on a learned driving decision model, and the first preset decision model can be obtained by training based on a reinforcement learning algorithm, wherein the reinforcement learning algorithm can be a reinforcement learning algorithm such as a Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm or a Deep Deterministic Policy Gradient (DDPG) algorithm. The second preset decision model can be one of a model-based vehicle lateral control model, a vehicle longitudinal control model, and a vehicle lateral and longitudinal integrated control model. For example, the vehicle lateral control model can be a preview angle model, the vehicle longitudinal control model can be an intelligent driver model (IDM) or the proportional integral model, and the vehicle lateral and longitudinal integrated control model can be a model predictive control model (MPC). For example, for the longitudinal control task, the intelligent driver model is selected as the second preset decision model. The selection result of the vehicle lateral and longitudinal control is related to the current control task of the vehicle.

[0074] Specifically, the terminal can obtain a first driving action decision result based on environmental information and a first preset decision model. The terminal can select a corresponding driving decision model according to the lateral and longitudinal control state of the vehicle. For a vehicle with only longitudinal control, the terminal can input environmental data into the longitudinal control model of the vehicle to obtain a second driving action decision result; for a vehicle with only lateral control, the terminal can input environmental data into the lateral control model of the vehicle to obtain a second driving action decision result; for a vehicle with integrated lateral and longitudinal control, the terminal can input environmental data into the integrated lateral and longitudinal control model of the vehicle to obtain a second driving action decision result; in addition, the terminal can also use deceleration, lane change and other command operations output by other decision algorithms.

[0075] Exemplarily, when the vehicle performs a longitudinal control task, the terminal obtains a first driving action decision result by means of reinforcement learning, converts environmental information into a state and inputs it into the current strategy of reinforcement learning to obtain the first driving action decision result. The terminal obtains a second driving action by inputting environmental information into a vehicle longitudinal control model (such as an intelligent driver model).

[0076] In this embodiment, by selecting different decision models in different scenarios, the accuracy of driving action decision results can be improved, and the safety of passengers can be further guaranteed in dangerous situations.

[0077] In an exemplary embodiment, the driving decision method further includes:

[0078] Calculate difference data between the driving decision and the first driving action decision result; determine target optimization parameters of the first preset decision model based on the difference data; update the first preset decision model based on the target optimization parameters to obtain the updated first preset decision model.

[0079] The difference data may represent the difference between the driving decision and the action instruction of the first driving action decision result, for example, the difference between deceleration, acceleration or steering, etc. The target optimization parameter may be the optimization parameter of the first preset decision model of the next state.

[0080] Specifically, the terminal may subtract the first driving action decision result from the driving decision to obtain a subtraction result, and take the absolute value of the subtraction result to obtain difference data. The terminal may calculate the target optimization parameter of the first preset decision model based on the difference data. The terminal may update the model parameters of the first preset decision model based on the target optimization parameter to obtain an updated first preset decision model.

[0081] In this embodiment, the target optimization parameters of the first preset decision model are determined by using the difference data between the driving decision and the first driving action decision result, so as to guide the first preset decision model to learn to make the optimal action decision in the actual scenario and accelerate the learning rate of reinforcement learning.

[0082] In an exemplary embodiment, determining a target optimization parameter of a first preset decision model based on the difference data includes:

[0083] Calculate the product value of the risk perception parameter and the difference data, and determine the product value as the first driving action decision result and the decision difference data of the driving decision; calculate the state action value function of the driving decision adopted under the environmental information; determine the objective function based on the state action value function and the decision difference data; update the parameter gradient of the first preset decision model according to the objective function and the data information in the experience replay pool, and obtain the target optimization parameters of the first preset decision model.

[0084] Among them, the value range of the risk perception parameter is related to the specific application scenario. The state action value function can reflect the predicted return of the action made according to the driving decision under the current environmental information. The experience replay pool can be used to store a series of data results generated by the vehicle in the process of interacting with the environment. Each experience usually includes the vehicle's environmental information, the vehicle's decision action, the reward feedback obtained by the vehicle from the environment after executing the decision action, the vehicle's next state, and whether it is terminated.

[0085] Specifically, the terminal can determine the risk perception parameter based on the current application scenario, and calculate the product value of the risk perception parameter and the difference data, and determine the product value as the decision difference value between the first driving action decision and the driving decision. The terminal can calculate the state action value function based on the preset action value function (Q function), the current environmental information, and the driving decision adopted under the current environmental information. The terminal can obtain the total objective function based on the optimization objectives of risk perception parameters, maximizing the state action value function, and minimizing the difference data. The terminal can calculate the vector of all partial derivatives of the objective function relative to the initial model parameters through the gradient calculation tool, and determine the vector as the gradient value of the initial model parameters. The terminal can calculate the mean of the objective function for the selected state-action pair based on the batch size of the state-action pairs selected in the batch. The terminal can calculate the gradient value relative to the initial model parameters based on the mean of the objective function, multiply the gradient value by the learning rate, and perform gradient update on the initial model parameters to obtain the target optimization parameters of the first preset decision model. The specific formula of the target optimization parameter can be:

[0086]

[0087] Wherein, θ is the parameter of the first preset decision model; β is the learning rate; is the gradient value of the initial model parameters; B is the number of state-action pairs in the experience replay pool; is the state action value function of driving decision-making under environmental information; s is the current state (environmental information), which is specifically defined as a multidimensional vector containing environmental information, π θ is the decision currently being executed (driving decision), which is specifically defined as the mapping relationship from the state s multidimensional vector to the action, λ is the risk perception parameter; φ1 is the reinforcement learning action network parameter.

[0088] The state action value function can represent the action in state s according to the strategy π θ The predicted reward for the chosen action.

[0089] When the risk perception parameter is 1, the corresponding vehicle is in a high-risk state, and the objective function is to maximize the state value function + minimize the decision difference data. The objective function is the optimization goal of the combination of the two; when the risk perception parameter is 0, the corresponding vehicle is in a low-risk state, and the objective function is to maximize the action value function.

[0090] when a=a RL When , the specific formula of the target optimization parameter can be:

[0091]

[0092] At this point, it can be understood that the first preset decision model is in a normal reinforcement learning update process.

[0093] In this embodiment, the target optimization parameters are calculated by using the difference data to guide the first preset decision model to learn to make the optimal action decision in the actual scenario, thereby accelerating the learning rate of reinforcement learning.

[0094] In an exemplary embodiment, the driving decision method further includes:

[0095] Collect sample physiological signals under preset dangerous scenarios; extract features of the sample physiological signals to obtain sample physiological signal features; train based on the sample physiological signal features and human subjective risk labels to obtain a preset risk perception model.

[0096] The preset danger scenario may be a simulated known risk scenario. The human subjective risk label may be a high risk label or a low risk label. The preset risk perception model may adopt a classification model or other supervised learning model. The preset risk perception model may characterize the mapping relationship between physiological signal characteristics and human subjective risk labels.

[0097] Specifically, the terminal can obtain sample physiological signals collected by the sensor equipment in preset dangerous scenarios and test sites based on the driving simulator and the real car. The terminal can use the time domain extraction tool to extract the time domain features of the sample physiological signals; use the Fourier transform algorithm to convert the time domain features into frequency domain features; extract the time series features of the sample physiological signals based on the deep learning network, and determine the time domain features, frequency domain features and time series features as sample physiological signal features. The terminal can input the sample physiological signal features into the human subjective risk label. In the human subjective risk label, based on the combination of the collected sample physiological signal features and the human subjective risk label data, a data set of the mapping relationship between the sample physiological signal features and the human subjective risk label is constructed, and the machine learning model is trained to perform classification analysis to obtain a preset risk perception model.

[0098] In addition, the terminal can preprocess the sample physiological signal to obtain a preprocessed sample physiological signal. Specifically, the preprocessing process may include: the terminal uses a filter to filter the sample physiological signal to obtain a filtered sample physiological signal; the terminal can collect the sample physiological signal in a resting state, and use the sample physiological signal in a resting state to perform baseline correction on the filtered sample physiological signal to obtain a baseline-corrected sample physiological signal. The terminal can use a motion artifact removal algorithm to remove motion artifacts from the baseline-corrected sample physiological signal to obtain a preprocessed sample physiological signal. The terminal can perform feature extraction on the preprocessed sample physiological signal.

[0099] In this embodiment, a preset risk perception model is trained through sample physiological signals to facilitate the perception of risk situations and provide guidance for determining accurate driving decisions.

[0100] In an exemplary embodiment, Figure 2 As shown, a driving decision-making method is provided, which may include two parts: a risk perception module based on the occupant status and a guided reinforcement learning module based on the occupant risk perception. The risk perception module based on the occupant status includes: collecting blood oxygen signals based on the blood oxygen monitoring device, preprocessing blood oxygen signals, extracting features, and training a preset risk perception model. The reinforcement learning module based on occupant risk perception includes: collecting blood oxygen signals, preprocessing blood oxygen signals, extracting features, risk perception, collecting environmental information, reinforcement learning, and monitoring correction. Monitoring correction includes two sub-steps: correcting decision results and guiding reinforcement learning.

[0101] The risk perception module based on the occupant status may specifically include: in a small number of common high-level autonomous driving scenarios with risks, the terminal obtains sample blood oxygen signals of the occupants collected by the blood oxygen detection equipment; the terminal preprocesses the sample blood oxygen signals to obtain preprocessed sample blood oxygen signals; the terminal extracts features of the preprocessed sample blood oxygen signals to obtain sample blood oxygen signal features, pairs the sample blood oxygen signal features with human subjective risk labels to construct a data set, and trains a machine learning model based on this data set to obtain human subjective risk labels.

[0102] The reinforcement learning module based on occupant risk perception may specifically include: in a high-level autonomous driving scenario with unknown risks, the terminal may obtain a blood oxygen signal, preprocess the blood oxygen signal to obtain a preprocessed blood oxygen signal, and the terminal may extract features of the preprocessed blood oxygen signal to obtain blood oxygen signal features; the terminal may input the blood oxygen signal features into a preset risk perception model for risk perception to obtain risk perception parameters; the terminal may obtain environmental information, input the environmental information into an intelligent driver model (a second preset decision model) to obtain the decision result of the intelligent driver model (a second driving action decision result), and input the environmental information into a reinforcement learning model (a first preset decision model) to obtain the decision result of the reinforcement learning model (a first driving action decision result); when the risk perception parameter indicates that the risk is high, the decision result of the intelligent driver model is used as the driving decision; otherwise, the decision result of the reinforcement learning model is used as the driving decision. The terminal may introduce the deviation between the driving decision and the decision result of the reinforcement learning model into the optimization target in the reinforcement learning model for reinforcement learning and correction, and correct the reinforcement learning model.

[0103] Among them, in order to remove the influence of factors such as breathing, heart rate, blood pressure fluctuations, Mayer wave noise, and movement on the result noise, the sample signal preprocessing process can specifically include: filtering the data with a filter, performing baseline correction based on the data collected at rest, and removing motion artifacts using the time derivative distribution repair (TDDR) algorithm. In addition, the original data dimension is relatively large, and feature extraction and dimensionality reduction operations are required.

[0104] In one example, the occupant's risk perception is extracted based on the blood oxygen signal, and the occupant's subjective risk perception result is used as the judgment condition. That is, when the occupant feels that the risk is high, the autonomous driving strategy (driving decision) is selected as the first driving action decision result; when the occupant feels that the risk is low, the reinforcement learning action is not intervened. At the same time, in order to effectively utilize the decision results of the intelligent driver model when the occupant feels that the risk is high, the reinforcement learning is timely guided to correct the wrong actions in this scenario, accelerate the learning rate of reinforcement learning, and introduce the deviation between the decision results of the intelligent driver model and the output results of reinforcement learning into the optimization target of the reinforcement learning action network to accelerate the learning rate of reinforcement learning.

[0105] Combined with fNIRS technology, the physiological information of the occupants is used as the basis for risk judgment of autonomous driving, and combined with the guided reinforcement learning algorithm, a safer, more humane, fast and reliable autonomous driving training method is achieved. The present invention further applies fNIRS technology to autonomous driving driving decisions. The embodiments of this application have high reference and practical value in the field of intelligent decision-making of high-level autonomous driving systems.

[0106] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0107] Based on the same inventive concept, the embodiment of the present application also provides a driving decision device for implementing the driving decision method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more driving decision device embodiments provided below can refer to the limitations on the driving decision method above, and will not be repeated here.

[0108] In an exemplary embodiment, Figure 3 As shown, a driving decision device 30 is provided, comprising: a first determination module 31, a second determination module 32 and a third determination module 33, wherein:

[0109] A first determination module 31, used to determine a first driving action decision result and a second driving action decision result based on environmental information and a preset decision model;

[0110] A second determination module 32, configured to determine risk perception parameters based on a preset risk perception model and physiological signal characteristics;

[0111] The third determination module 33 is used to determine the driving decision based on the risk perception parameter, the first driving action decision result and the second driving action decision result.

[0112] In an exemplary embodiment, the third determination module 33 is used to determine the second driving action decision result as the driving decision if the risk perception parameter is the first parameter; if the risk perception parameter is the second parameter, determine the first driving action decision result as the driving decision.

[0113] In an exemplary embodiment, the first determination module 31 is used to obtain a first driving action decision result based on environmental information and a first preset decision model; and to obtain a second driving action decision result based on the environmental information and the second preset decision model.

[0114] In an exemplary embodiment, the first determination module 31 is also used to calculate the difference data between the driving decision and the first driving action decision result; determine the target optimization parameters of the first preset decision model based on the difference data; and update the first preset decision model based on the target optimization parameters to obtain an updated first preset decision model.

[0115] In an exemplary embodiment, the first determination module 31 is also used to calculate the product value of the risk perception parameter and the difference data, and determine the product value as the first driving action decision result and the decision difference data of the driving decision; calculate the state action value function of the driving decision adopted under the environmental information; determine the objective function based on the state action value function and the decision difference data; update the parameter gradient of the first preset decision model according to the objective function and the data information in the experience replay pool to obtain the target optimization parameters of the first preset decision model.

[0116] In one of the embodiments, a training module is used to collect sample physiological signals under a preset danger scenario; perform feature extraction on the sample physiological signals to obtain sample physiological signal features; perform training based on the sample physiological signal features and human subjective risk labels to obtain a preset risk perception model, wherein the preset risk perception model characterizes the mapping relationship between physiological signal features and human subjective risk labels.

[0117] Each module in the above-mentioned driving decision-making device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0118] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 4As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (Near Field Communication, NFC) or other technologies. When the computer program is executed by the processor, a driving decision method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0119] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0120] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0121] Determining a first driving action decision result and a second driving action decision result based on the environmental information and a preset decision model;

[0122] Determine risk perception parameters based on the preset risk perception model and physiological signal characteristics;

[0123] A driving decision is determined based on the risk perception parameter, the first driving action decision result and the second driving action decision result.

[0124] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0125] The first driving action decision result and the second driving action decision result determine a driving decision, including:

[0126] If the risk perception parameter is the first parameter, determining the second driving action decision result as the driving decision;

[0127] If the risk perception parameter is the second parameter, the first driving action decision result is determined as the driving decision.

[0128] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0129] Obtaining a first driving action decision result based on the environmental information and the first preset decision model;

[0130] Based on the environmental information and the second preset decision model, a second driving action decision result is obtained.

[0131] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0132] Calculating difference data between the driving decision and the first driving action decision result;

[0133] Determining a target optimization parameter of the first preset decision model based on the difference data;

[0134] The first preset decision model is updated based on the target optimization parameter to obtain an updated first preset decision model.

[0135] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0136] Calculating a product value of a risk perception parameter and the difference data, and determining the product value as decision difference data between the first driving action decision result and the driving decision;

[0137] Calculating a state action value function for adopting the driving decision under the environmental information;

[0138] An objective function is determined based on the state-action value function and the decision difference data; and a parameter gradient of a first preset decision model is updated according to the objective function and data information in an experience replay pool to obtain target optimization parameters of the first preset decision model.

[0139] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0140] Collect sample physiological signals under preset dangerous scenarios;

[0141] Extracting features of the sample physiological signal to obtain features of the sample physiological signal;

[0142] Based on the sample physiological signal characteristics and human subjective risk labels, training is performed to obtain a preset risk perception model, wherein the preset risk perception model represents the mapping relationship between the physiological signal characteristics and the human subjective risk labels.

[0143] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0144] Determining a first driving action decision result and a second driving action decision result based on the environmental information and a preset decision model;

[0145] Determine risk perception parameters based on the preset risk perception model and physiological signal characteristics;

[0146] A driving decision is determined based on the risk perception parameter, the first driving action decision result and the second driving action decision result.

[0147] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0148] Determining a first driving action decision result and a second driving action decision result based on the environmental information and a preset decision model;

[0149] Determine risk perception parameters based on the preset risk perception model and physiological signal characteristics;

[0150] A driving decision is determined based on the risk perception parameter, the first driving action decision result and the second driving action decision result.

[0151] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0152] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0153] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0154] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A driving decision method, characterized in that: The method comprises: Determining a first driving action decision result and a second driving action decision result based on the environmental information and a preset decision model; Determine risk perception parameters based on the preset risk perception model and physiological signal characteristics; A driving decision is determined based on the risk perception parameter, the first driving action decision result and the second driving action decision result.

2. The method according to claim 1, characterized in that The determining of the driving decision based on the risk perception parameter, the first driving action decision result and the second driving action decision result includes: If the risk perception parameter is a first parameter, determining the second driving action decision result as the driving decision, and the first parameter indicates that the vehicle is in a high-risk state; If the risk perception parameter is a second parameter, the first driving action decision result is determined as the driving decision, and the second parameter indicates that the vehicle is in a low-risk state.

3. The method according to claim 1, characterized in that The preset decision model includes a first preset decision model and a second preset decision model, wherein the first preset decision model is obtained by training based on a reinforcement learning model, and the second preset decision model includes one of a vehicle lateral control model, a vehicle longitudinal control model, and a vehicle lateral and longitudinal integrated control model, and the first driving action decision result and the second driving action decision result are determined based on the environmental information and the preset decision model, including: Obtaining a first driving action decision result based on the environmental information and the first preset decision model; Based on the environmental information and the second preset decision model, a second driving action decision result is obtained.

4. The method according to claim 3, characterized in that The method further comprises: Calculating difference data between the driving decision and the first driving action decision result; Determining a target optimization parameter of the first preset decision model based on the difference data; The first preset decision model is updated based on the target optimization parameter to obtain an updated first preset decision model.

5. The method according to claim 4, characterized in that The determining the target optimization parameter of the first preset decision model based on the difference data includes: Calculating a product value of a risk perception parameter and the difference data, and determining the product value as decision difference data between the first driving action decision result and the driving decision; Calculating a state action value function for adopting the driving decision under the environmental information; Determining an objective function based on the state-action-value function and the decision difference data; The parameter gradient of the first preset decision model is updated according to the objective function and the data information in the experience replay pool to obtain the target optimization parameters of the first preset decision model.

6. The method according to claim 1, characterized in that The method further comprises: Collect sample physiological signals under preset dangerous scenarios; Extracting features of the sample physiological signal to obtain features of the sample physiological signal; Based on the sample physiological signal characteristics and human subjective risk labels, training is performed to obtain a preset risk perception model, where the preset risk perception model characterizes the mapping relationship between the physiological signal characteristics and the human subjective risk labels.

7. A driving decision-making device, characterized in that: The device comprises: A first determination module, used to determine a first driving action decision result and a second driving action decision result based on environmental information and a preset decision model; A second determination module is used to determine risk perception parameters based on a preset risk perception model and physiological signal characteristics; The third determination module is used to determine the driving decision based on the risk perception parameter, the first driving action decision result and the second driving action decision result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images