An electroencephalogram emergency control system and method in an autonomous driving scenario

Abstract

The application relates to an electroencephalogram emergency control system and method in an automatic driving scenario, which comprises a vehicle sensor module, an electroencephalogram acquisition module, a comprehensive cognition module, a judgment module and an electroencephalogram processing module; the vehicle sensor module is used for collecting road condition information and vehicle condition information in real time; the electroencephalogram acquisition module is used for collecting electroencephalogram signals in real time; the comprehensive cognition module is used for forming comprehensive cognition of the environment in the sensing range, the vehicle self state and the driving internal scene according to the road condition information and the vehicle condition information; the judgment module is used for judging whether the vehicle meets a dangerous situation triggering condition based on the comprehensive cognition; then when the dangerous situation triggering condition is met, it is judged whether there is an action electroencephalogram input; the electroencephalogram processing module is used for obtaining electroencephalogram control information after analyzing the action electroencephalogram, and adjusting the vehicle driving posture according to the electroencephalogram control information. The application provides a safer guarantee measure for the existing unmanned vehicle, and effectively reduces the frequent occurrence of accidents.

Description

Technical Field

[0001] This invention relates to the field of traffic driving technology, and in particular to an emergency control system and method for brainwaves in autonomous driving scenarios. Background Technology

[0002] Bioelectric phenomena are one of the fundamental characteristics of life activities, and brain waves are one type of bioelectricity. When humans engage in thought processes, the bioelectric signals generated in the brain are called brain waves. These brain wave signals can be measured and studied using sensors placed on the scalp. Since the last century, through the study of brain wave signals, people have increasingly deepened and enriched their understanding of the brain.

[0003] Current autonomous driving platforms lack effective solutions for situations such as icy or snow-covered roads, hailstorms, heavy fog, muddy roads, construction site obstacles, and pedestrians suddenly entering the driving lane. In these extreme situations, relying solely on path planning and radar detection is clearly unreliable. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides an emergency control system and method for brainwaves in autonomous driving scenarios, which solves the technical problem that autonomous driving cannot handle dangerous situations such as hail or fog.

[0006] (II) Technical Solution

[0007] To achieve the above objectives, the main technical solutions adopted by the present invention include:

[0008] In a first aspect, embodiments of the present invention provide an emergency control system for brainwaves in an autonomous driving scenario, comprising: a vehicle sensor module, a brainwave acquisition module, a comprehensive cognition module, a judgment module, and a brainwave processing module;

[0009] The vehicle sensor module is used to collect road condition information and vehicle condition information in real time and send them to the comprehensive cognition module;

[0010] The EEG acquisition module is used to acquire EEG signals of several people in the vehicle in real time and send them to the EEG processing module.

[0011] The comprehensive cognition module is used to form a comprehensive cognition of the environment, vehicle status and driving interior within the sensing range based on the road condition information and vehicle condition information through multi-sensor data fusion, and send it to the judgment module.

[0012] The judgment module is used to determine whether the vehicle meets the emergency triggering conditions based on the comprehensive cognition; and then, when the emergency triggering conditions are met, to determine whether there is action brainwave input, and to send the judgment result to the brainwave processing module.

[0013] The EEG processing module analyzes the judgment result. When there is an action EEG input, it intercepts the EEG signal within a preset time period starting from the first moment, obtains the raw wave data after preprocessing, and then analyzes the raw wave data to obtain EEG control information. Based on the EEG control information, it adjusts the vehicle's driving posture.

[0014] Optionally,

[0015] The vehicle sensor module includes lidar, millimeter-wave radar, ultrasonic radar, 3D sensor, ambient light sensor, and IMU sensor.

[0016] The EEG acquisition module includes several EEG sensors and Bluetooth / serial port components.

[0017] The EEG sensor is used to collect EEG signals from people in the vehicle and send them to the EEG processing module via the Bluetooth / serial port component.

[0018] The Bluetooth / serial port component is used for networking, enabling several EEG sensors to establish a one-to-many network with the EEG processing module through the Bluetooth / serial port component.

[0019] Optionally, the EEG acquisition device is an EEG helmet or EEG headband.

[0020] Optionally, the action brainwave is determined by whether the difference between the feature value of the brainwave signal at the first moment and the average feature value of the brainwave signal in the previous preset time period is less than a preset threshold.

[0021] If the value is less than the preset threshold, it is determined that there is no action EEG input;

[0022] If the value exceeds the preset threshold, it is determined that there is an EEG input indicating movement.

[0023] The characteristic values ​​include amplitude, frequency, and average potential.

[0024] Optionally, the system further includes a signal source processing module, used for:

[0025] Determine the number of people whose input brainwaves correspond to the input action;

[0026] If the number of personnel sources is equal to 1, then proceed to the process steps of the judgment module;

[0027] If the number of personnel sources is greater than 1

[0028] The first decision-maker is selected based on the average contribution rate, field of view, and focus value of each person's location, and the brainwave signal of the first decision-maker is used as the signal input of the judgment module.

[0029] or,

[0030] Based on the average contribution rate, field of view, and focus value of each person, weights are assigned to the EEG signals from different people. Each EEG signal is then fused with its corresponding weight, and the fused value of the EEG signals is used as the signal input to the judgment module.

[0031] Optionally,

[0032] The average contribution rate of the location is:

[0033]

[0034]

[0035] in, p represents the historical contribution rate of position i during the j-th instance of human brain takeover of driving. j This is the jth evaluation of human brain takeover driving. The brainwave signal of the person in position i during the j-th brain-controlled driving session;

[0036] The field of view includes the first field of view from the driver's seat, the second field of view from the front passenger seat, the third field of view from the left-side window passenger seat, and the fourth field of view from the right-side window passenger seat; the first field of view ≥ the second field of view > the third field of view = the fourth field of view.

[0037] The focus value is extracted from the electroencephalogram (EEG) signal and ranges from 0 to 100.

[0038] Optionally, the EEG processing module includes a preprocessing module, an empirical pattern decomposition module, a feature extraction module, and a classification module;

[0039] The preprocessing module is used to preprocess the EEG signals captured within a preset time period starting from the first moment to obtain the original wave signals.

[0040] The empirical mode decomposition module is used to decompose the original wave signal based on the empirical mode decomposition algorithm to obtain multiple IMF components.

[0041] The feature extraction module is used to extract the mean and balance coefficient of the IMF components as feature vectors;

[0042] The classification module is used to classify feature vectors based on the fuzzy C-means clustering algorithm, and obtain EEG control information based on the classification results;

[0043] The balance coefficient is: σ 2 Let Variance be the variance of the IMF components. This represents the average value of the IMF components.

[0044] Optionally, the system also includes a hazard matching module, a route adjustment module, and an early warning module;

[0045] When the conditions for triggering a hazard are met but no action EEG signal is input, the hazard matching module will match the hazard with hazard response plans in a hazard response plan library established based on prior experience.

[0046] When a corresponding emergency response plan is matched, the path adjustment module applies the emergency response plan to adjust the vehicle's driving posture.

[0047] When no corresponding emergency response plan is found, the warning module sends a warning signal to the people in the vehicle and resolves the emergency based on the brainwave feedback signals generated by the people in response to the warning signal.

[0048] Optionally, the system further includes a process evaluation and optimization module, which is used for:

[0049] After the car's driving posture is adjusted, a human brain takeover driving process and human brain takeover driving evaluation are generated.

[0050] Based on the evaluation of human brain takeover driving, the human brain takeover driving process was optimized, and the optimized human brain takeover driving process was saved to the emergency response plan database.

[0051] Secondly, embodiments of the present invention provide a method for emergency brainwave control in an autonomous driving scenario, comprising:

[0052] Real-time collection of road and vehicle condition information;

[0053] Real-time acquisition of brainwave signals from several people in the vehicle;

[0054] Based on the road condition information and the vehicle condition information, a comprehensive understanding of the environment, vehicle status, and driving interior is formed through multi-sensor data fusion.

[0055] Based on the comprehensive understanding described above, determine whether the vehicle meets the conditions for triggering a hazard.

[0056] Then, when the conditions for triggering a dangerous situation are met, it is determined whether there is any brainwave input related to movement;

[0057] When there is an input of brainwaves, brainwave signals within a preset time period are captured from the first moment. After preprocessing, raw wave data is obtained. Then, the raw wave data is analyzed to obtain brainwave control information, and the vehicle driving posture is adjusted according to the brainwave control information.

[0058] (III) Beneficial Effects

[0059] The beneficial effects of this invention are as follows: The brainwave emergency control system of this invention, by employing a brainwave module, a comprehensive cognition module, and a judgment module, compared with existing autonomous driving systems, can adjust the vehicle's driving posture in a timely manner based on the collected brainwave signals of the driver or passenger when the driver or passenger uses the autonomous driving function, in response to emergency situations or extreme road conditions within the driver's field of vision. This provides a safer guarantee for existing autonomous driving systems and effectively reduces the frequency of autonomous driving accidents. Attached Figure Description

[0060] Figure 1 A schematic diagram of the composition of an emergency brainwave control system in an autonomous driving scenario provided by the present invention;

[0061] Figure 2 A flowchart illustrating an emergency brainwave control method for autonomous driving scenarios provided by the present invention;

[0062] Figure 3 The workflow of the application platform for an emergency brainwave control method in an autonomous driving scenario provided by the present invention. Detailed Implementation

[0063] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0064] An embodiment of the present invention proposes an emergency brainwave control system for autonomous driving scenarios, characterized in that, as Figure 1As shown, the system includes: a vehicle sensor module, an EEG acquisition module, a comprehensive cognition module, a judgment module, and an EEG processing module. The vehicle sensor module is used to collect real-time road and vehicle condition information. The EEG acquisition module is used to collect real-time EEG signals. The comprehensive cognition module is used to form a comprehensive understanding of the environment within the sensing range, the vehicle's own state, and the driving interior based on road and vehicle condition information, where the driving interior refers to the environment inside the vehicle and the position and posture of the passengers. The judgment module is used to determine whether the vehicle meets the emergency triggering conditions based on the comprehensive cognition; and then, when the emergency triggering conditions are met, to determine whether there is action EEG input. The EEG processing module is used to capture EEG signals within a preset time period starting from the first moment when action EEG input occurs, and obtain raw wave data through preprocessing, where the first moment is the moment of action EEG input. The raw wave data is then analyzed to obtain EEG control information, and the vehicle's driving posture is adjusted according to the EEG control information.

[0065] The brainwave emergency control system of the present invention, by employing a brainwave module, a comprehensive cognition module, and a judgment module, can, compared with existing autonomous driving, adjust the vehicle's driving posture in a timely manner based on the collected brainwave signals of the driver or passenger when the driver or passenger is using the autonomous driving function, in response to emergency situations or extreme road conditions within the driver's field of vision. This provides a safer guarantee for existing autonomous driving and effectively reduces the frequency of autonomous driving accidents.

[0066] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and that the scope of the present invention can be fully conveyed to those skilled in the art.

[0067] Specifically, the vehicle sensor module includes lidar, millimeter-wave radar, ultrasonic radar, 3D sensors, and IMU sensors, as well as temperature sensors and ambient light sensors.

[0068] The EEG module includes several EEG acquisition devices and a Bluetooth / serial port component. The EEG acquisition devices collect EEG signals from people in the vehicle and transmit them to the EEG processing module via the Bluetooth / serial port component. The Bluetooth / serial port component is used for networking, enabling several EEG acquisition channels to establish a one-to-many network with the EEG processing module. Preferably, the EEG acquisition device is an EEG helmet / EEG headband.

[0069] Furthermore, the action EEG is determined by whether the difference between the characteristic value of the EEG signal at the first moment and the average characteristic value of the EEG signal in the previous preset time period is less than a preset threshold.

[0070] If the value is less than the preset threshold, it is determined that there is no action EEG input.

[0071] If the value exceeds a preset threshold, it is determined that there is an input of action-related brainwaves.

[0072] Among them, the characteristic values ​​include amplitude, frequency and average potential, and the average characteristic value is the average value of the characteristic values ​​of the EEG signal at each moment within the previous preset time period.

[0073] Next, the system also includes a signal source processing module, which is used for:

[0074] By determining the number of people corresponding to the input brainwave signals.

[0075] If the number of personnel sources is equal to 1, then proceed to the process steps of the judgment module.

[0076] If the number of personnel sources is greater than 1, the first decision-maker is selected based on the average contribution rate of each person's location, field of view, and focus value. The brainwave signal of the first decision-maker is used as the signal input of the judgment module.

[0077] or,

[0078] If the number of people from different sources is greater than 1, the correlation between multiple EEG signals is determined.

[0079] The system assigns weights to the EEG signals from different individuals based on their average contribution rate, field of view, and focus level. It then fuses each EEG signal with its corresponding weight and uses the fused EEG signal value as the input signal to the judgment module.

[0080] In the above screening or weighting process, the individual's reaction time and driving stress level can be taken into account to make the input signal more accurate. The reaction time is the sequence of the EEG signals emitted by each individual within a preset time period starting from the first moment. The driving stress level is determined based on the collected physiological signs and mental state of each individual, ranging from 0 to 100.

[0081] Furthermore, the average contribution rate based on location is:

[0082]

[0083]

[0084] in, p represents the historical contribution rate of position i during the j-th instance of human brain takeover of driving. j This is the jth evaluation of human brain takeover driving. The brainwave signal of the person in position i during the j-th brain takeover of driving.

[0085] The field of view includes the first field of view from the driver's seat, the second field of view from the front passenger seat, the third field of view from the left-side window passenger seat, and the fourth field of view from the right-side window passenger seat; the first field of view ≥ the second field of view > the third field of view = the fourth field of view.

[0086] Attention score is extracted from EEG signals and ranges from 0 to 100.

[0087] Electroencephalogram (EEG) signals include delta waves, theta waves, alpha waves, and beta waves;

[0088] The frequency range of delta waves in brainwaves is 0.5–4 Hz; amplitude: 20–200 μV.

[0089] The frequency range of theta waves in the EEG is 4–7 Hz; amplitude: 20–150 μV;

[0090] The frequency range of alpha waves in the EEG is 8–12 Hz; amplitude: 20–100 μV;

[0091] The frequency range of beta waves in the EEG is 12–30 Hz; amplitude: 5–20 μV.

[0092] Attention level is measured by beta wave power / theta wave power.

[0093] Subsequently, the EEG processing module includes a preprocessing module, an experience pattern decomposition module, a feature extraction module, and a classification module.

[0094] The preprocessing module is used to preprocess the EEG signals captured within a preset time period starting from the first moment to obtain the raw wave signals. Generally, EEG signals are recorded from the surface of the scalp; these signals are relatively weak and easily affected by external and subjective factors. The acquired EEG signals contain various types of artifacts, such as pulse, electrooculography (EOG), and electromyography (EMG). Therefore, artifact subtraction is used to analyze and identify eye movement and EMG artifacts.

[0095] The empirical pattern decomposition module is used to decompose the original wave signal based on the empirical pattern decomposition algorithm to obtain multiple IMF components; the feature extraction module is used to extract the mean and balance coefficient of the IMF components as feature vectors; the classification module is used to classify the feature vectors based on the fuzzy C-means clustering algorithm, and obtain EEG control information based on the classification results.

[0096] Wherein, the balance coefficient is σ 2Let Variance be the variance of the IMF components. This represents the average value of the IMF components.

[0097] Empirical pattern decomposition (EMD) is an effective method for processing nonlinear data. It decomposes a signal into multiple IMF components, each containing different frequency components. The feature values ​​of the IMF components are extracted, and then fuzzy C-means clustering is used for classification. The classification effect and efficiency are both good.

[0098] Next, the system also includes a hazard matching module, a path adjustment module, and an early warning module. When the hazard triggering conditions are met but no action brainwave signal is input, the hazard matching module matches the hazard with hazard response plans in a hazard response plan library established based on prior experience. When a corresponding hazard response plan is matched, the path adjustment module applies the hazard response plan to adjust the vehicle's driving posture. When no corresponding hazard response plan is matched, the early warning module sends an early warning signal to the occupants of the vehicle and resolves the hazard based on the brainwave feedback signals generated by the occupants in response to the early warning signal.

[0099] Furthermore, the system also includes a process evaluation and optimization module, which is used for:

[0100] After the vehicle's driving posture is adjusted, a human brain takeover driving process and a human brain takeover driving evaluation are generated. Based on the human brain takeover driving evaluation, the human brain takeover driving process is optimized, and the optimized human brain takeover driving process is saved to the emergency response plan database.

[0101] Figure 2 This is a flowchart illustrating an emergency brainwave control method for autonomous driving scenarios provided by the present invention, as shown below. Figure 2 As shown, the present invention also provides a brainwave emergency control method in an autonomous driving scenario, applied to the above-mentioned brainwave emergency control system, comprising:

[0102] S1. Real-time collection of road and vehicle condition information.

[0103] S2. Real-time acquisition of brainwave signals from several people in the vehicle.

[0104] S3: Based on road and vehicle information, it forms a comprehensive understanding of the environment, vehicle status, and driving interior through multi-sensor data fusion.

[0105] S4. Based on comprehensive understanding, determine whether the vehicle meets the conditions for triggering a hazard.

[0106] S5. Then, when the conditions for triggering the emergency are met, determine whether there is any brainwave input of movement.

[0107] S6. When there is an action brainwave input, the brainwave signal within a preset time period is intercepted from the current moment, the raw wave data is obtained after preprocessing, and then the raw wave data is analyzed to obtain brainwave control information, and the vehicle driving posture is adjusted according to the brainwave control information.

[0108] Furthermore, the method also includes:

[0109] F61. When the conditions for triggering a hazard are met but there is no input of action EEG signals, the hazard will be matched with the hazard response plan in the hazard response plan library established based on prior experience.

[0110] F62. When a corresponding emergency response plan is matched, apply the emergency response plan to adjust the vehicle's driving posture.

[0111] F63. If no corresponding emergency response plan is found, send a warning signal to the people in the vehicle and resolve the emergency based on the brainwave feedback signals generated by the people in response to the warning signal.

[0112] Furthermore, prior to step S6, the following steps are also included:

[0113] By determining the number of people corresponding to the input brainwave signals.

[0114] F61. If the number of personnel sources is equal to 1, then proceed to the process steps of the judgment module.

[0115] F62a. If the number of personnel sources is greater than 1, the first decision-maker is selected based on the average contribution rate of each person's location, field of view, and focus value. The brainwave signal of the first decision-maker is used as the signal input of the judgment module.

[0116] or,

[0117] F62b: If the number of personnel sources is greater than 1, perform correlation judgment on multiple EEG signals.

[0118] The system assigns weights to the EEG signals from different individuals based on their average contribution rate, field of view, and focus level. It then fuses each EEG signal with its corresponding weight and uses the fused EEG signal value as the input signal to the judgment module.

[0119] Further, step S6 includes:

[0120] S61. Preprocess the EEG signals captured within a preset time period starting from the first moment to obtain the original wave signals.

[0121] S62. The original wave signal is decomposed based on the empirical mode decomposition algorithm to obtain multiple IMF components.

[0122] S63. Extract the mean and balance coefficient of the IMF components as feature vectors.

[0123] S64. The feature vectors are classified based on the fuzzy C-means clustering algorithm, and the brainwave control information is obtained based on the classification results.

[0124] Furthermore, after step S6, the following steps are also included:

[0125] After the car's driving posture is adjusted, the human brain takeover driving process and human brain takeover driving evaluation are generated.

[0126] Based on the evaluation of human brain takeover driving, the human brain takeover driving process was optimized, and the optimized human brain takeover driving process was saved to the emergency response plan database.

[0127] In addition, the present invention also provides a computer-readable medium storing computer-executable instructions thereon, which, when executed by a processor, implement the steps of an emergency brainwave control method in an autonomous driving scenario for an unmanned vehicle as described above.

[0128] In a specific embodiment, the present invention is applied to a specific simulated driving application platform. This experimental platform includes the following components: several EEG sensors, an Arduino UNO board, an L298N DC motor driver board, a Bluetooth module, two 130 micro motors with wheels, a gyroplane, a power bank, a line-following module, several bolts, and several DuPont wires. The chassis, support plate, and gear structure are all 3D printed. The specific construction process is as follows: First, 123Design is used to complete the 3D model of the simulated vehicle device, and Cure is used to slice the model. Second, the above components are assembled according to the 3D model to form the simulated vehicle device, and the peripheral circuits of the EEG module are set up and debugged. Simultaneously, a 3D engine is used to load the 3D vehicle model and arrange the VR scene, linking the control of the simulated vehicle device with that of the 3D vehicle model to form a complete VR system.

[0129] Figure 3 The workflow of the application platform for the brainwave emergency control method in an autonomous driving scenario provided by this invention is as follows: Figure 3 As shown, in the VR scene, the simulated vehicle device is controlled by human brain signals. At the same time, the collected vehicle condition information and simulated road condition information are sent to the MCU via Bluetooth module. After receiving the data packet, it is stored in the MCU's receiving buffer. When the simulated vehicle sends a dangerous situation, the operator receives sensory stimulation and sends action brainwaves to the MCU. Then, through a series of data processing, brainwave control signals are obtained.

[0130] The data packet parsing and extraction process is mainly designed and processed by the microcontroller through a program. The specific processing is as follows: (1) Data packet parsing process. During the data packet parsing process, the simulated vehicle main control device receives the data in the form of a data stream. At the same time as receiving the data packet, the meaning structure is also parsed, and the data verification process is added; (2) Extraction of valid data. After the data packet is parsed, the original wave data is obtained in sequence. Then, the direction of the simulated vehicle device is controlled by extracting the feature values ​​of the EEG signal. Then, the dual-wheel DC motor is driven by the motor to control the movement and deceleration of the simulated vehicle device.

[0131] In summary, this invention provides an emergency control system, method, and medium for brainwaves in autonomous driving scenarios. This invention obtains comprehensive cognitive information encompassing environmental and / or vehicle information, the vehicle's own motion state, and the driving interior through multi-sensor fusion technology. Based on this comprehensive cognitive information, it determines the vehicle's current state. Then, when the conditions for triggering a hazard are met, it analyzes the brainwave control information of the brainwave signals at that time and adjusts the vehicle's driving posture according to this brainwave control information to resolve vehicle or road hazard situations.

[0132] When encountering emergencies or extreme road conditions that cannot be handled or take too long to process while driving autonomously, this invention can effectively help existing autonomous vehicles avoid or handle such complex dangers, thus effectively improving the safety of existing autonomous driving.

[0133] Since the systems / devices described in the above embodiments of the present invention are systems / devices used to implement the methods of the above embodiments of the present invention, those skilled in the art can understand the specific structure and modifications of the systems / devices based on the methods described in the above embodiments of the present invention, and therefore will not be repeated here. All systems / devices used in the methods of the above embodiments of the present invention fall within the scope of protection of the present invention.

[0134] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0135] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.

[0136] It should be noted that any reference numerals placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In claims that enumerate several means, several of these means may be embodied by the same hardware. The use of the terms first, second, third, etc., is merely for convenience of expression and does not indicate any order. These terms can be understood as part of the component names.

[0137] Furthermore, it should be noted that in the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0138] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the claims should be interpreted to include both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0139] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, then this invention should also include these modifications and variations.

Claims

1. A brainwave emergency control system for autonomous driving scenarios, characterized in that, The brainwave emergency control system is applied to scenarios of hail and fog within the human field of vision. The brainwave emergency control system includes: vehicle sensor module, brainwave acquisition module, comprehensive cognition module, judgment module, brainwave processing module, signal source processing module, hazard matching module, path adjustment module, and early warning module. The vehicle sensor module is used to collect road condition information and vehicle condition information in real time and send them to the comprehensive cognition module; The EEG acquisition module is used to acquire EEG signals of several people in the vehicle in real time and send them to the EEG processing module. The comprehensive cognition module is used to form a comprehensive cognition of the environment, vehicle status and driving interior within the sensing range based on the road condition information and vehicle condition information through multi-sensor data fusion, and send it to the judgment module. The judgment module is used to determine whether the vehicle meets the emergency triggering conditions based on the comprehensive cognition; and then, when the emergency triggering conditions are met, to determine whether there is action brainwave input, and to send the judgment result to the brainwave processing module. The EEG processing module analyzes the judgment result. When there is an action EEG input, it intercepts the EEG signal within a preset time period starting from the first moment, obtains the raw wave data after preprocessing, and then analyzes the raw wave data to obtain EEG control information. Based on the EEG control information, it adjusts the vehicle's driving posture. The signal source processing module is used for: Determine the number of people whose input brainwaves correspond to the input action; If the number of personnel sources is equal to 1, then proceed to the process steps of the judgment module; If the number of personnel is greater than 1, the first decision-maker is selected based on the average contribution rate of each person's location, field of view, focus value, reaction time of the person's actions, and driving stress value. The brainwave signal of the first decision-maker is used as the signal input of the judgment module. When the conditions for triggering a hazard are met but no action EEG signal is input, the hazard matching module will match the hazard with hazard response plans in a hazard response plan library established based on prior experience. When a corresponding emergency response plan is matched, the path adjustment module applies the emergency response plan to adjust the vehicle's driving posture. The warning module is used to send a warning signal to the people in the vehicle when no corresponding emergency response plan is found, and to resolve the emergency based on the brainwave feedback signals generated by the people in response to the warning signal. The average contribution rate of the location is: ； ； in, For the first j During brain takeover of driving i Historical contribution rate of location p j For the first j Evaluation of human brain takeover driving. In the first j During the second brain takeover driving, in i EEG signals of people at the location; The field of view includes the first field of view from the driver's seat, the second field of view from the front passenger seat, the third field of view from the left-side window passenger seat, and the fourth field of view from the right-side window passenger seat; the first field of view ≥ the second field of view > the third field of view = the fourth field of view. The focus value is extracted from the electroencephalogram (EEG) signal and is measured by the β wave power / theta wave power in the EEG signal. The focus value ranges from 0 to 100. The action brainwave is determined by whether the difference between the feature value of the brainwave signal at the first moment and the average feature value of the brainwave signal in the previous preset time period is less than a preset threshold. The action reaction time is the sequence of the emission of brainwaves of each person within a preset time period starting from the first moment: The driving stress value is determined based on the collected physiological signs and mental state of each person, and the driving stress value ranges from 0 to 100.

2. The brainwave emergency control system for autonomous driving scenarios as described in claim 1, characterized in that, The vehicle sensor module includes lidar, millimeter-wave radar, ultrasonic radar, 3D sensor, ambient light sensor, and IMU sensor. The EEG acquisition module includes several EEG sensors and Bluetooth / serial port components. The EEG sensor is used to collect EEG signals from people in the vehicle and send them to the EEG processing module via the Bluetooth / serial port component. The Bluetooth / serial port component is used for networking, enabling several EEG sensors to establish a one-to-many network with the EEG processing module through the Bluetooth / serial port component.

3. A brainwave emergency control system for autonomous driving scenarios as described in claim 1 or 2, characterized in that, The brainwave acquisition device is a brainwave helmet or brainwave headband.

4. The brainwave emergency control system for autonomous driving scenarios as described in claim 1, characterized in that, If the difference between the characteristic value of the EEG signal at the first moment and the average characteristic value of the EEG signal in the previous preset time period is less than a preset threshold, it is determined that there is no action EEG input. If the difference between the characteristic value of the EEG signal at the first moment and the average characteristic value of the EEG signal in the previous preset time period is greater than the preset threshold, it is determined that there is an action EEG input. The characteristic values ​​include amplitude, frequency, and average potential.

5. The brainwave emergency control system for autonomous driving scenarios as described in claim 1, characterized in that, The EEG processing module includes a preprocessing module, an empirical pattern decomposition module, a feature extraction module, and a classification module; The preprocessing module is used to preprocess the EEG signals captured within a preset time period starting from the first moment to obtain the original wave signals. The empirical mode decomposition module is used to decompose the original wave signal based on the empirical mode decomposition algorithm to obtain multiple IMF components. The feature extraction module is used to extract the mean and balance coefficient of the IMF components as feature vectors; The classification module is used to classify feature vectors based on the fuzzy C-means clustering algorithm, and obtain EEG control information based on the classification results; The balance coefficient is: , Let Variance be the variance of the IMF components. This represents the average value of the IMF components.

6. The brainwave emergency control system for autonomous driving scenarios as described in claim 1, characterized in that, The system also includes a process evaluation and optimization module, which is used for: After the car's driving posture is adjusted, a human brain takeover driving process and human brain takeover driving evaluation are generated. Based on the evaluation of human brain takeover driving, the human brain takeover driving process was optimized, and the optimized human brain takeover driving process was saved to the emergency response plan database.

7. A brainwave emergency control method for autonomous driving scenarios, applied to the brainwave emergency control system as described in any one of claims 1-6, characterized in that, include: Real-time collection of road and vehicle condition information; Real-time acquisition of brainwave signals from several people in the vehicle; Based on the road condition information and the vehicle condition information, a comprehensive understanding of the environment, vehicle status, and driving interior is formed through multi-sensor data fusion. Based on the comprehensive understanding described above, determine whether the vehicle meets the conditions for triggering a hazard. Then, when the conditions for triggering a dangerous situation are met, it is determined whether there is any brainwave input related to movement; When there is an input of brainwaves, brainwave signals within a preset time period are captured from the first moment. After preprocessing, raw wave data is obtained. Then, the raw wave data is analyzed to obtain brainwave control information, and the vehicle driving posture is adjusted according to the brainwave control information.

Images