A vehicle control method and device, vehicle and storage medium

Abstract

The embodiment of the application discloses a vehicle control method, device, vehicle and storage medium, and belongs to the technical field of vehicle control, mainly comprising the following steps: collecting the driver's brain electrical signals, environment state information and vehicle state information of the vehicle in operation in real time; analyzing the driver's brain electrical signals in real time to obtain real-time driver's mental load and real-time driver's control decision; analyzing the environment state information and vehicle state information of the vehicle in real time to obtain real-time vehicle intelligent control decision; calculating the real-time proportion of the driver's control right and the real-time proportion of the vehicle intelligent control right according to the real-time driver's mental load, the real-time driver's control decision and the real-time vehicle intelligent control decision; and distributing the control right to the driver in real time, and intelligently controlling the vehicle according to the real-time proportion of the vehicle intelligent control right. The application can improve the traffic efficiency, improve the traffic safety, and improve the driving experience of the driver.

Description

Technical Field

[0001] This invention relates to the field of vehicle control technology, and in particular to a vehicle control method, device, vehicle, and storage medium. Background Technology

[0002] Prolonged driving significantly increases the mental workload of drivers, potentially leading to road accidents. Human-vehicle collaborative driving can reduce this mental workload and improve vehicle safety. While existing human-vehicle collaborative driving solutions can reduce driver workload by setting autonomous driving parameters, they do not consider the driver's physiological state, resulting in a poor driver experience and increasing the safety risks caused by vehicle control system errors. Summary of the Invention

[0003] This invention provides a vehicle control method, device, vehicle, and storage medium that can avoid safety risks caused by driver fatigue and system errors, improve traffic efficiency, enhance traffic safety, and improve the driver's driving experience.

[0004] In a first aspect, embodiments of the present invention provide a vehicle control method, comprising: real-time acquisition of driver's electroencephalogram (EEG) signals, environmental state information, and vehicle state information of a vehicle in operation; real-time analysis of the driver's EEG signals to obtain real-time driver mental load and real-time driver control decisions; real-time analysis of the vehicle's environmental state information and vehicle state information to obtain real-time vehicle intelligent control decisions; calculation of the real-time percentage of driver control rights and the real-time percentage of vehicle intelligent control rights during the control of the vehicle based on the real-time driver mental load, the real-time driver control decisions, and the real-time vehicle intelligent control decisions; real-time allocation of control rights to the driver based on the real-time percentage of driver control rights, and intelligent control of the vehicle based on the real-time percentage of vehicle intelligent control rights.

[0005] Secondly, embodiments of the present invention provide a vehicle control device, comprising: a data acquisition module for real-time acquisition of driver's electroencephalogram (EEG) signals, environmental state information, and vehicle state information of a vehicle in operation; a first analysis module for real-time analysis of the driver's EEG signals to obtain real-time driver mental load and real-time driver control decisions; a second analysis module for real-time analysis of the vehicle's environmental state information and vehicle state information to obtain real-time vehicle intelligent control decisions; a control rights percentage calculation module for calculating, based on the real-time driver mental load, the real-time driver control decisions, and the real-time vehicle intelligent control decisions, the real-time percentage of driver control rights and the real-time percentage of vehicle intelligent control rights during the control of the vehicle; and a control rights allocation and control module for real-time allocation of control rights to the driver based on the real-time percentage of driver control rights and intelligent control of the vehicle based on the real-time percentage of vehicle intelligent control rights.

[0006] Thirdly, embodiments of the present invention also provide a vehicle, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the vehicle control method as described in any of the embodiments of the present invention.

[0007] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the vehicle control method as described in any of the embodiments of the present invention.

[0008] The present invention provides a vehicle control method, device, vehicle, and storage medium that allocates the real-time proportion of driver control and vehicle intelligent control based on the driver's mental workload in a human-vehicle co-driving scenario. This can avoid safety risks caused by excessive driver fatigue, as well as safety risks caused by system errors, thereby improving traffic efficiency, traffic safety, and the driver's driving experience. Attached Figure Description

[0009] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic flowchart of a vehicle control method provided in an embodiment of the present invention;

[0011] Figure 2This is another schematic flowchart of the vehicle control method provided in this embodiment of the invention;

[0012] Figure 3 This is another schematic flowchart of the vehicle control method provided in this embodiment of the invention;

[0013] Figure 4 This is a schematic diagram of a vehicle control device provided in an embodiment of the present invention;

[0014] Figure 5 This is a structural schematic diagram of a vehicle provided in an embodiment of the present invention. Detailed Implementation

[0015] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0016] Modern road traffic is increasingly congested, and prolonged driving significantly increases drivers' mental workload, raising the risk of accidents. Human-vehicle collaborative driving is a crucial means of reducing driver workload and improving vehicle safety. It organically integrates human decision-making capabilities with vehicle perception capabilities, effectively reducing driver workload while maintaining driver control, minimizing traffic accidents caused by human error, and improving traffic safety and efficiency, thus achieving safer and more reliable driving. Existing human-vehicle collaborative driving solutions use LiDAR and cameras installed in the vehicle to collect information about the surrounding environment. While this environmental big data guides the setting of autonomous driving parameters and reduces driver workload, it doesn't consider the driver's physiological state, resulting in a poor driver experience and increasing the safety risks caused by vehicle control system errors.

[0017] The present invention provides a vehicle control method, device, vehicle, and storage medium that allocates the real-time proportion of driver control and vehicle intelligent control based on the driver's mental workload in a human-vehicle co-driving scenario. This can avoid safety risks caused by excessive driver fatigue, as well as safety risks caused by system errors, thereby improving traffic efficiency, traffic safety, and the driver's driving experience.

[0018] Figure 1This is a schematic flowchart illustrating a vehicle control method provided in an embodiment of the present invention. This method can be executed by a vehicle control device provided in this embodiment, which can be implemented using software and / or hardware. In a specific embodiment, the device can be integrated into a vehicle. The following embodiments will illustrate this using the integration of the device into a vehicle client as an example. (Reference) Figure 1 The method may specifically include the following steps:

[0019] Step 101 involves real-time acquisition of the driver's electroencephalogram (EEG) signals, environmental status information, and vehicle status information during operation. This facilitates the analysis of the driver's EEG signals to determine real-time driver mental workload and control decisions.

[0020] Specifically, the process of real-time acquisition of the EEG signals of the driver of the vehicle in operation may include: having the driver of the vehicle wear a 32-lead EEG helmet to acquire the driver's current EEG signals.

[0021] Specifically, the process of collecting environmental status information of vehicles in operation includes: using the vehicle-mounted sensing system to collect the current environmental status information and current vehicle status information of the vehicles.

[0022] Specifically, the aforementioned vehicle-mounted perception system may include camera equipment, radar equipment, positioning equipment, speed measurement equipment, etc.

[0023] Specifically, the environmental status information of the vehicle collected in real time includes: road markings ahead of the vehicle's direction of travel, traffic light information, obstacle information, weather information, visibility information, and the vehicle's position.

[0024] Specifically, the real-time vehicle status information collected includes: the vehicle's current direction of travel, speed, braking status, energy consumption status, etc.

[0025] Step 102 involves real-time analysis of the driver's EEG signals to obtain real-time driver mental load and real-time driver control decisions. This allows for the calculation of the real-time percentage of driver control and vehicle intelligent control during vehicle control based on real-time driver mental load and real-time driver control decisions.

[0026] In an optional specific embodiment of the present invention, the process of performing real-time analysis of the driver's EEG signal to obtain the real-time driver's mental load and real-time driver's control decision includes: filtering out interference signals from the driver's EEG signal to obtain interference-free EEG signal.

[0027] Specifically, after the driver's EEG signal is acquired in real time, interference signals above 60Hz can be filtered out, and power frequency interference can be filtered out to obtain the aforementioned interference-free EEG signal.

[0028] In an optional specific embodiment of the present invention, the process of obtaining real-time driver mental load and real-time driver control decision by real-time analysis of driver's EEG signals includes: extracting multiple EEG signal rhythms from interference-free EEG signals based on a frequency slice wavelet transform algorithm.

[0029] Specifically, the above-mentioned multiple EEG signal rhythms can also be extracted based on other rhythm extraction methods.

[0030] Specifically, EEG signals are usually formed by a complex combination of multiple frequency waveforms. The dominant waveform can be identified by visual inspection to determine that the EEG signal is in a certain rhythm or frequency band, or spectral analysis can be used for identification.

[0031] Specifically, the aforementioned multiple EEG signal rhythms can include: delta-band EEG signals, theta-band EEG signals, alpha-band EEG signals, beta-band EEG signals, and gamma-band EEG signals. Among them, the delta-band (1-4Hz) is associated with deep relaxation and restorative sleep. Irregular delta wave movements are closely related to cognitive difficulties and problems maintaining consciousness; theta-band (4-8Hz) is extremely prominent in adults experiencing frustration, depression, or mental illness, and is commonly found in people in a trance or hypnotic state; the alpha-band (8-14Hz) can help you calm down and promote deeper relaxation and satisfaction, and this rhythm is most pronounced when a person is awake, quiet, and has their eyes closed; the beta-band (14-31Hz) appears when a person is mentally stressed, emotionally excited, or agitated, and when a person wakes up from a nightmare, the original slow wave rhythm can be immediately replaced by this rhythm; the gamma-band (31-51Hz) involves gamma waves in addition to participating in healthy cognitive functions, and also in processing more complex tasks. Gamma waves are crucial for learning, memory, and processing; they are used as a combined tool for our senses to process new information.

[0032] In an optional specific embodiment of the present invention, the process of obtaining real-time driver mental load and real-time driver control decision by real-time analysis of driver's EEG signals includes: analyzing multiple EEG signal rhythms to obtain real-time driver mental load and real-time driver control decision.

[0033] Specifically, different EEG signal rhythms show different trends as the driver's mental load and control decisions change. Therefore, it is necessary to first extract the EEG signal rhythms and analyze them separately to obtain the aforementioned real-time driver mental load and real-time driver control decisions.

[0034] Optionally, the process of analyzing the above multiple EEG signal rhythms to obtain the real-time driver's mental load includes: calculating the rhythm power of the first load-related rhythm among the multiple EEG signal rhythms that is related to mental load to obtain the first rhythm power, and calculating the rhythm power of the second load-related rhythm among the multiple EEG signal rhythms that is related to mental load to obtain the second rhythm power; the first rhythm power increases with the increase of mental load, and the second rhythm power decreases with the increase of mental load; the ratio of the first rhythm power to the second rhythm power is calculated to obtain the load correlation ratio, and the real-time driver's mental load is determined based on the load correlation ratio.

[0035] Specifically, the aforementioned first load-related rhythm and the aforementioned second load-related rhythm can also be EEG signal rhythms that exhibit other changing trends with changes in mental workload.

[0036] In an optional embodiment of the present invention, the first load-related rhythm is a theta-band EEG signal, and the second load-related rhythm is a beta-band EEG signal. Studies have shown that when the brain load increases, the theta wave rhythm power P... θ Increased β-wave rhythm power P β Reduced. Therefore, this embodiment of the invention uses P. θ / P β This indicator is used to judge the driver's mental workload; the higher the ratio, the higher the mental workload.

[0037] Specifically, the load-related ratio is directly used as the aforementioned driver's mental load.

[0038] In an optional specific embodiment of the present invention, the process of determining the real-time driver's mental load based on the load correlation ratio includes: determining the driver's real-time mental load level based on the load correlation ratio and a preset range of mental load levels with multiple load correlation ratio thresholds as boundaries, and determining the driver's real-time mental load level as the real-time driver's mental load.

[0039] Specifically, the multiple mental workload level intervals mentioned above, which are bounded by multiple workload-related ratio thresholds, can be set based on empirical data.

[0040] Specifically, the aforementioned ranges of mental workload levels, defined by multiple load-related ratio thresholds, can be set based on an individual's load-related ratio.

[0041] Optionally, the multiple mental workload level intervals with the above-mentioned multiple workload-related ratio thresholds as boundaries can also be obtained based on empirical data or in other ways.

[0042] In an optional embodiment of the present invention, before determining the driver's real-time mental workload level based on the load correlation ratio and a preset range of mental workload levels bounded by multiple load correlation ratio thresholds, the method further includes: obtaining the minimum value of the load correlation ratio corresponding to the driver's reported lowest mental workload state and the maximum value of the load correlation ratio corresponding to the driver's reported highest mental workload state; and dividing the range from the minimum value of the load correlation ratio to the maximum value of the load correlation ratio into a predetermined number of sub-ranges to obtain a plurality of mental workload level ranges bounded by multiple load correlation ratio thresholds.

[0043] Specifically, to accommodate individual differences among drivers, this invention adopts an equal conversion algorithm based on maximum and minimum values. That is, the load correlation ratio of the driver under the lowest brain load state and the load correlation ratio under the highest brain load state are pre-entered according to the driver's second gear description, and the brain load level interval is calculated based on these two as benchmarks. This facilitates the matching of brain load level intervals based on the real-time measured load correlation ratio, and the interval that falls into is converted into the corresponding brain load level.

[0044] Specifically, the aforementioned predetermined number can be 10.

[0045] In an optional specific embodiment of the present invention, the aforementioned process of analyzing multiple EEG signal rhythms to obtain real-time driver control decisions includes: extracting features from multiple EEG signal rhythms using an energy matrix image mapping method to obtain EEG image features.

[0046] Optionally, a power spectral density algorithm can be used to map the above-mentioned multiple EEG signal rhythms into an energy matrix image as the features of the above-mentioned EEG feature image, so as to contain and reveal more underlying information of EEG signals in the form of feature images.

[0047] In an optional specific embodiment of the present invention, the aforementioned process of analyzing multiple EEG signal rhythms to obtain real-time driver control decisions includes: inputting EEG image features into a trained driver control decision prediction model, and using the driver control decision prediction model to output real-time driver control decisions.

[0048] Specifically, the aforementioned real-time driver control decisions may include various control decisions such as moving forward, reversing, turning left, turning right, and stopping.

[0049] Specifically, the aforementioned control decision prediction model can be a neural network model, such as a long short-term memory network model.

[0050] Specifically, the aforementioned EEG feature images can be fed into a Long Short-Term Memory (LSTM) network model for classification to obtain the current driver's driving intention. The LTM network outputs five numbers from 1 to 5, representing the driver's five driving intentions: forward, backward, left turn, right turn, and stop.

[0051] Specifically, the aforementioned control decision prediction model is obtained by training the neural network with training EEG signals collected from the driver's EEG signals when the driver makes the aforementioned multiple control decisions on the driving simulator, or by training the neural network with training EEG signals collected from the driver's EEG signals when the driver imagines the behavior of making the aforementioned multiple control decisions.

[0052] For example, the brain signals of a driver in a driving simulator in a state of controlling forward movement or in a state of imagining a forward movement command are collected and the brain signals in this state are marked with the number 1. These signals are then used to train a neural network to obtain the aforementioned control decision prediction model.

[0053] Step 103 involves real-time analysis of the vehicle's environmental and vehicle status information to obtain real-time intelligent vehicle control decisions. This allows for the calculation of the real-time percentage of driver control and vehicle intelligent control during the vehicle control process based on these decisions, and further vehicle control based on these decisions and the real-time percentage of vehicle intelligent control.

[0054] Optionally, the process of obtaining real-time vehicle intelligent control decisions by real-time analysis of the vehicle's environmental state information and vehicle state information includes: extracting effective features of environmental state information from the vehicle's environmental state information and extracting effective features of vehicle state information from the vehicle state information; fusing the effective features of environmental state information and vehicle state information into a complete feature vector for intelligent control decisions; and obtaining the vehicle intelligent control decisions based on the feature vector for intelligent control decisions.

[0055] Specifically, the aforementioned vehicle intelligent control decisions can include: moving forward, reversing, turning left, turning right, and stopping, which can be represented by five numbers from 1 to 5 in sequence.

[0056] Step 104: Based on the real-time driver's mental workload, real-time driver control decisions, and real-time vehicle intelligent control decisions, the real-time percentage of driver control rights and the real-time percentage of vehicle intelligent control rights are calculated during the vehicle control process. This facilitates real-time allocation of driver control rights based on the real-time percentage of driver control rights and intelligent vehicle control based on the real-time percentage of vehicle intelligent control rights.

[0057] In an optional specific embodiment of the present invention, the process of calculating the real-time proportion of driver control rights and the real-time proportion of vehicle intelligent control rights during the process of controlling the vehicle based on the real-time driver mental load, real-time driver control decisions and real-time vehicle intelligent control decisions includes: calculating the real-time proportion of driver control rights and the real-time proportion of vehicle intelligent control rights based on a preset real-time proportion of control rights calculation formula and the real-time proportion of driver control rights and the real-time proportion of vehicle intelligent control rights based on the real-time driver mental load.

[0058] Optionally, if the real-time driver control decision matches the real-time vehicle intelligent control decision, the real-time proportion of driver control and the real-time proportion of vehicle intelligent control are calculated based on the preset real-time proportion calculation formula and the real-time driver mental load; otherwise, the real-time proportion of driver control and the real-time proportion of vehicle intelligent control are calculated based on the confidence level of the real-time driver control decision calculated based on the real-time driver mental load.

[0059] Specifically, based on the aforementioned real-time driver mental workload and following the "human-in-the-loop" allocation principle, the real-time weighting of driver control and vehicle intelligent control can be calculated using the following formula:

[0060]

[0061] In the formula, w1 represents the real-time percentage of driver control, w2 represents the real-time percentage of vehicle intelligent control, and x represents the real-time driver mental workload.

[0062] Step 105: Real-time allocation of driver control rights is performed based on the real-time percentage of driver control rights, and intelligent vehicle control is implemented based on the real-time percentage of vehicle intelligent control rights. This allows for shared driving between the driver and the vehicle, based on the real-time allocation of driver and vehicle intelligent control rights according to the driver's mental workload. This avoids safety risks caused by driver fatigue and system errors, improving traffic efficiency, enhancing traffic safety, and improving the driver's experience.

[0063] The following section further introduces vehicle control methods, such as... Figure 2 As shown, the vehicle control method of the present invention may include the following steps:

[0064] Step 201: Real-time collection of driver's EEG signals, environmental status information, and vehicle status information of the vehicle in operation.

[0065] Step 202: Perform real-time analysis of the driver's EEG signals to obtain the real-time driver's mental load and real-time driver control decisions.

[0066] Step 203: Perform real-time analysis of the vehicle's environmental status information and vehicle status information to obtain real-time vehicle intelligent control decisions.

[0067] Step 204: Based on the real-time driver's mental load and the accuracy of the driver control decision prediction model in predicting driver control decisions, calculate the confidence level of the real-time driver control decision based on the fuzzy membership function.

[0068] Specifically, a driver control decision confidence model based on fuzzy membership functions can be established to assess the reliability of the driver's control decisions by comprehensively considering the driver's real-time mental workload. The following fuzzy membership function can be established:

[0069]

[0070] In the formula, x represents the real-time mental workload of the driver, and r represents the accuracy of the control decision prediction model in predicting the driver's control decisions.

[0071] Optionally, the fuzzy membership degrees calculated from the above fuzzy membership function are normalized by transforming them using a base-10 logarithm, which yields the confidence level y of the driver's driving decision. The transformation formula is as follows:

[0072] .

[0073] Step 205: If the real-time driver control decision does not match the real-time vehicle intelligent control decision, determine whether the confidence level of the real-time driver control decision is less than a preset confidence level threshold. If it is less, set the real-time proportion of vehicle intelligent control rights to 100% and the real-time proportion of driver control rights to 0; otherwise, set the real-time proportion of driver control rights to 100% and the real-time proportion of vehicle intelligent control rights to 0.

[0074] Specifically, the aforementioned preset confidence threshold can be set based on empirical data, and can be set to 50%.

[0075] Optionally, the analysis and calculation of real-time driver mental load, the analysis and calculation of real-time driver control decisions, and the analysis and calculation of real-time vehicle intelligent control decisions are performed simultaneously.

[0076] In alternative specific examples of the invention, such as Figure 3As shown, the aforementioned process of calculating the real-time percentage of driver control and the real-time percentage of vehicle intelligent control during vehicle control based on real-time driver mental load, real-time driver control decisions, and real-time vehicle intelligent control decisions includes: judging whether the real-time driver control decisions and real-time vehicle intelligent control decisions are consistent; if the real-time driver control decisions and real-time vehicle intelligent control decisions are consistent, then the real-time percentage of driver control and the real-time percentage of vehicle intelligent control are calculated based on the preset real-time percentage calculation formula and the real-time driver mental load; otherwise, judging whether the confidence level of the real-time driver control decisions is less than 50%; if it is less than 50%, then the real-time percentage of vehicle intelligent control is set to 100% and the real-time percentage of driver control is set to 0; otherwise, the real-time percentage of driver control is set to 100% and the real-time percentage of vehicle intelligent control is set to 0.

[0077] In an optional specific embodiment of the present invention, the driver determines that the vehicle should turn left based on the current road environment. The real-time driver control decision is calculated as 3 (left turn) based on the driver's EEG signal analysis. The confidence level of the current decision, calculated based on the driver's EEG signal analysis and real-time driver mental load, is 60%. Simultaneously, the vehicle makes an intelligent control decision based on environmental and vehicle status information, outputting a real-time vehicle intelligent control decision as 1 (forward). The real-time driver control decision and the real-time vehicle intelligent control decision are compared. Since the real-time driver control decision and the real-time vehicle intelligent control decision are inconsistent, and the confidence level of the real-time driver control decision is greater than a preset confidence threshold of 50%, the real-time driver control decision takes precedence, and the driver has complete control over the vehicle's operation.

[0078] Step 206: Real-time allocation of control rights to the driver based on the real-time percentage of driver control rights, and intelligent control of the vehicle based on the real-time percentage of vehicle intelligent control rights.

[0079] This invention determines the confidence level of real-time driver control decisions based on the driver's real-time mental load. By utilizing the confidence level of real-time driver control decisions, when the driver's mental load is high, such as after a long period of driving and when fatigued, the vehicle intelligent control system takes full control of the vehicle's operation to avoid potential safety hazards caused by mental fatigue. Conversely, when the driver's mental load is low, the system can take full control of the vehicle itself to avoid safety hazards caused by system errors, while also improving the driving experience.

[0080] Figure 4 This is a structural diagram of a vehicle control device provided in an embodiment of the present invention. This device is suitable for executing the vehicle control method provided in an embodiment of the present invention. Figure 3As shown, the device may specifically include:

[0081] The data acquisition module 401 is used to collect real-time driver brainwave signals, environmental status information, and vehicle status information of a vehicle in operation. This module can facilitate the analysis of driver brainwave signals to obtain real-time driver mental load and real-time driver control decisions.

[0082] The first analysis module 402 is used to perform real-time analysis of the driver's electroencephalogram (EEG) signals to obtain real-time driver mental load and real-time driver control decisions. This module can calculate the real-time percentage of driver control and the real-time percentage of vehicle intelligent control during the vehicle control process based on the real-time driver mental load and real-time driver control decisions.

[0083] The second analysis module 403 is used to perform real-time analysis of the vehicle's environmental state information and vehicle state information to obtain real-time vehicle intelligent control decisions. This module can calculate the real-time proportion of driver control rights and the real-time proportion of vehicle intelligent control rights during the vehicle control process based on the vehicle control decisions, and further control the vehicle according to the vehicle control decisions and the real-time proportion of vehicle intelligent control rights.

[0084] The control rights percentage calculation module 404 is used to calculate the real-time percentage of driver control rights and the real-time percentage of vehicle intelligent control rights during the vehicle control process, based on the real-time driver's mental load, real-time driver control decisions, and real-time vehicle intelligent control decisions. This module facilitates real-time allocation of driver control rights based on the real-time percentage of driver control rights and intelligent control of the vehicle based on the real-time percentage of vehicle intelligent control rights.

[0085] The control allocation and control module 405 is used to allocate control rights to the driver in real time based on the real-time proportion of driver control rights, and to perform intelligent control of the vehicle based on the real-time proportion of vehicle intelligent control rights. Building upon the operations of modules 401-404, this module allocates control rights based on the real-time proportion of driver and vehicle intelligent control rights according to the driver's mental workload, enabling shared driving between the driver and the vehicle. This avoids safety risks caused by driver fatigue and system errors, improves traffic efficiency and safety, and enhances the driver's experience.

[0086] In an optional embodiment of the present invention, the vehicle control device of the present invention further includes a confidence calculation module, which is used to calculate the confidence of the real-time driver control decision based on a fuzzy membership function, according to the real-time driver mental load and the accuracy of the driver control decision prediction model when predicting the driver control decision.

[0087] In an optional specific embodiment of the present invention, the first analysis module 402 can be specifically used to filter out interference signals from the driver's EEG signal to obtain an interference-free EEG signal; extract multiple EEG signal rhythms from the interference-free EEG signal based on the frequency slice wavelet transform algorithm; and analyze the multiple EEG signal rhythms to obtain the real-time driver's mental load and the real-time driver's control decision.

[0088] In an optional embodiment of the present invention, the first analysis module 402 is specifically used to calculate the rhythm power of the first load-related rhythm among multiple EEG signal rhythms that is related to mental load, and to calculate the rhythm power of the second load-related rhythm among multiple EEG signal rhythms that is related to mental load, to obtain the second rhythm power; the first rhythm power increases with the increase of mental load, and the second rhythm power decreases with the increase of mental load; the ratio of the first rhythm power to the second rhythm power is calculated to obtain the load-related ratio, and the real-time driver's mental load is determined based on the load-related ratio.

[0089] In an optional embodiment of the present invention, the first analysis module 402 can be specifically used to determine the driver's real-time mental load level based on the load correlation ratio and a preset range of mental load levels with multiple load correlation ratio thresholds as boundaries, and to determine the driver's real-time mental load level as the real-time driver mental load.

[0090] In an optional embodiment of the present invention, the first analysis module 402 can be specifically used to obtain the minimum value of the load-related ratio corresponding to the lowest mental load state reported by the driver and the maximum value of the load-related ratio corresponding to the highest mental load state reported by the driver; and to divide the interval from the minimum value of the load-related ratio to the maximum value of the load-related ratio into a predetermined number of sub-intervals to obtain multiple mental load level intervals with multiple load-related ratio thresholds as boundaries.

[0091] In an optional specific embodiment of the present invention, the first analysis module 402 can be specifically used to extract features from multiple EEG signal rhythms based on the energy matrix image mapping method to obtain EEG image features; input the EEG image features into a trained driver control decision prediction model, and use the driver control decision prediction model to output real-time driver control decisions.

[0092] The control rights percentage calculation module 404 is specifically used to calculate the real-time percentage of driver control rights and the real-time percentage of vehicle intelligent control rights based on the preset real-time control rights percentage calculation formula and the real-time driver mental load if the real-time driver control decision matches the real-time vehicle intelligent control decision; otherwise, it judges whether the confidence level of the real-time driver control decision is less than the preset confidence level threshold. If it is less, it sets the real-time percentage of vehicle intelligent control rights to 100% and the real-time percentage of driver control rights to 0; otherwise, it sets the real-time percentage of driver control rights to 100% and the real-time percentage of vehicle intelligent control rights to 0.

[0093] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is merely an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the functional modules described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0094] This invention also provides a vehicle, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the vehicle control method provided in any of the above embodiments.

[0095] This invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the vehicle control method provided in any of the above embodiments.

[0096] The following is for reference. Figure 5 It shows a schematic diagram of the structure of a computer system 500 suitable for implementing embodiments of the present invention in a vehicle. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0097] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage section 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the system 500. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0098] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 510 as needed so that computer programs read from it can be installed into storage section 508 as needed.

[0099] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs the functions defined above in the system of this invention.

[0100] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0101] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0102] The modules and / or units described in the embodiments of this invention can be implemented in software or hardware. The described modules and / or units can also be housed in a processor; for example, a processor can be described as including a data acquisition module, a first analysis module, a second analysis module, a control weight percentage calculation module, and a control weight allocation and control module. The names of these modules do not necessarily limit the module itself under certain circumstances.

[0103] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to: collect in real time the driver's electroencephalogram (EEG) signals, environmental state information, and vehicle state information of a vehicle in operation; analyze the driver's EEG signals in real time to obtain real-time driver mental load and real-time driver control decisions; analyze the environmental state information and vehicle state information of the vehicle in real time to obtain real-time vehicle intelligent control decisions; calculate, based on the real-time driver mental load, the real-time driver control decisions, and the real-time vehicle intelligent control decisions, the real-time percentage of driver control rights and the real-time percentage of vehicle intelligent control rights during the control of the vehicle; and allocate control rights to the driver in real time based on the real-time percentage of driver control rights, and perform intelligent control of the vehicle based on the real-time percentage of vehicle intelligent control rights.

[0104] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0105] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A vehicle control method, characterized in that, include: Real-time collection of driver brainwave signals, environmental status information, and vehicle status information of vehicles in operation; Real-time analysis of the driver's EEG signals is used to obtain the real-time driver's mental load, and real-time driver control decisions are obtained based on the driver control decision prediction model. Real-time vehicle intelligent control decisions are obtained by analyzing the environmental state information and vehicle state information of the vehicle in real time. If the real-time driver control decision matches the real-time vehicle intelligent control decision, then based on the preset real-time control right ratio calculation formula, the real-time ratio of driver control right and the real-time ratio of vehicle intelligent control right are calculated according to the real-time driver mental load. Otherwise, based on the real-time driver's mental load and the accuracy of the driver control decision prediction model in predicting the driver's control decision, the confidence level of the real-time driver control decision is calculated using the fuzzy membership function. The system determines whether the confidence level of the real-time driver control decision is less than a preset confidence level threshold. If it is less, the real-time percentage of the vehicle intelligent control right is set to 100%, and the real-time percentage of the driver control right is set to 0. Otherwise, the real-time percentage of the driver control right is set to 100%, and the real-time percentage of the vehicle intelligent control right is set to 0. The driver's control is allocated in real time based on the real-time percentage of the driver's control rights, and the vehicle is intelligently controlled based on the real-time percentage of the vehicle's intelligent control rights.

2. The vehicle control method according to claim 1, characterized in that, The process of performing real-time analysis of the driver's EEG signals to obtain real-time driver mental load and real-time driver control decisions includes: Interference signals in the driver's EEG signal are filtered out to obtain interference-free EEG signal; Multiple EEG rhythms were obtained by performing rhythm extraction on the interference-free EEG signal using a frequency slice wavelet transform algorithm; and The real-time driver's mental load and real-time driver's control decisions are obtained by analyzing the multiple EEG signal rhythms.

3. The vehicle control method according to claim 2, characterized in that, The process of analyzing the multiple EEG signal rhythms to obtain the real-time driver's mental load includes: The first rhythm power is obtained by calculating the rhythm power of the first load-related rhythm among the plurality of EEG signal rhythms that is related to mental workload, and the second rhythm power is obtained by calculating the rhythm power of the second load-related rhythm among the plurality of EEG signal rhythms that is related to mental workload; the first rhythm power increases with the increase of mental workload, and the second rhythm power decreases with the increase of mental workload; The load correlation ratio is obtained by calculating the ratio of the first rhythm power to the second rhythm power, and the real-time driver mental load is determined based on the load correlation ratio.

4. The vehicle control method according to claim 3, characterized in that, The process of determining the real-time driver mental load based on the load correlation ratio includes: Based on the load correlation ratio and a preset range of mental workload levels with multiple load correlation ratio thresholds as boundaries, the real-time mental workload level of the driver is determined, and the real-time mental workload level of the driver is defined as the real-time driver mental workload.

5. The vehicle control method according to claim 4, characterized in that, Before determining the driver's real-time mental workload level based on the load correlation ratio and multiple preset mental workload level intervals bounded by multiple load correlation ratio thresholds, the method further includes: Obtain the minimum value of the load-related ratio corresponding to the driver's reported lowest mental workload state and the maximum value of the load-related ratio corresponding to the driver's reported highest mental workload state; and The interval between the minimum and maximum values ​​of the load correlation ratio is divided into a predetermined number of sub-intervals to obtain multiple mental workload level intervals with multiple load correlation ratio thresholds as boundaries.

6. The vehicle control method according to claim 2, characterized in that, The process of analyzing the multiple EEG signal rhythms to obtain the real-time driver control decision includes: The method based on energy matrix image mapping is used to extract features from the multiple EEG signal rhythms to obtain EEG image features; The EEG image features are input into a trained driver control decision prediction model, and the real-time driver control decision is obtained by using the output of the driver control decision prediction model.

7. A vehicle control device for executing the vehicle control method as described in any one of claims 1-6, characterized in that, include: The data acquisition module is used to collect real-time data on the driver's electroencephalogram (EEG) signals, environmental status information, and vehicle status information of vehicles in operation. The first analysis module is used to perform real-time analysis of the driver's EEG signals to obtain the real-time driver's mental load and real-time driver's control decisions. The second analysis module is used to perform real-time analysis of the environmental state information and vehicle state information of the vehicle to obtain real-time vehicle intelligent control decisions. The control rights percentage calculation module is used to calculate the real-time percentage of driver control rights and the real-time percentage of vehicle intelligent control rights during the process of controlling the vehicle, based on the real-time driver mental load, the real-time driver control decision, and the real-time vehicle intelligent control decision. as well as The control allocation and control module is used to allocate control rights to the driver in real time according to the real-time proportion of the driver's control rights, and to perform intelligent control of the vehicle according to the real-time proportion of the vehicle's intelligent control rights.

8. A vehicle comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the vehicle control method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the vehicle control method as described in any one of claims 1 to 6.

Images