Vehicle control method and device based on mixed brain and muscle signals, equipment and medium

Abstract

The invention relates to the technical field of automatic driving, in particular to a vehicle control method and device based on mixed brain and muscle signals, equipment and a medium. According to the method, multiple brain and muscle signals of a target object and a mixed brain and muscle signal set composed of multiple electromyographic signals are obtained, and feature processing is conducted on the mixed brain and muscle signal set; and obtaining a target electroencephalogram feature and a target myoelectricity feature. And performing driving behavior detection on the target electroencephalogram features and the target myoelectricity features through a preset hybrid strategy to obtain behavior probability values corresponding to a plurality of preset driving behaviors, thereby determining the target driving behavior of the target object through the plurality of behavior probability values, and performing corresponding control on the vehicle. Therefore, the driving intention of the target object is analyzed and predicted more comprehensively by combining the signal advantages of the electroencephalogram signal and the electromyographic signal, so that the accuracy of driving behavior prediction and the reliability and safety of vehicle control are improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a vehicle control method, device, equipment, and medium based on hybrid brain-muscle signals. Background Art

[0002] With the rapid development of automobile technology, traffic accidents still cause safety and economic losses frequently. Advanced Driver Assistance Systems (ADAS) have emerged as the times require. They can assist drivers in various ways, especially in detecting driver status and driving behavior, and can effectively reduce the incidence of traffic accidents.

[0003] In the prior art, traditional driving behavior detection systems mainly rely on signals such as vehicle information, environmental information, and driver behavior information. However, the acquisition of these signals is limited, and it is difficult to accurately predict the driver's intentions, resulting in low efficiency in driving behavior detection and low reliability and safety of vehicle control. Summary of the invention

[0004] In view of the above problems, the present invention provides a vehicle control method, device, equipment, and medium based on hybrid brain-muscle signals to improve the accuracy of driving behavior detection and the reliability and safety of vehicle control.

[0005] In a first aspect, the present application provides a vehicle control method based on hybrid brain-muscle signals, the method comprising:

[0006] Acquire a mixed brain-muscle signal set of a target object, and perform feature processing on the mixed brain-muscle signal set to obtain target EEG features and target myoelectric features; the mixed brain-muscle signal set includes multiple EEG signals and multiple myoelectric signals of the target object;

[0007] Based on a preset hybrid strategy, driving behavior detection is performed on the target EEG feature and the target EMG feature to obtain a plurality of behavior probability values; each behavior probability value corresponds to a preset driving behavior;

[0008] Based on the multiple behavior probability values, a target driving behavior of the target object is determined, and based on the target driving behavior, the vehicle is controlled accordingly.

[0009] In a second aspect, the present application provides a vehicle control device based on mixed brain-muscle signals, the device comprising:

[0010] An acquisition unit is used to acquire a mixed brain-muscle signal set of a target object, and perform feature processing on the mixed brain-muscle signal set to obtain a target EEG feature and a target EMG feature; the mixed brain-muscle signal set includes a plurality of EEG signals and a plurality of EMG signals of the target object;

[0011] A detection unit, configured to perform driving behavior detection on the target EEG feature and the target EMG feature based on a preset hybrid strategy to obtain a plurality of behavior probability values; each behavior probability value corresponds to a preset driving behavior;

[0012] A control unit is used to determine a target driving behavior of the target object based on the multiple behavior probability values, and to control the vehicle accordingly based on the target driving behavior.

[0013] Optionally, when the preset mixing strategy is a feature mixing strategy, the detection unit is specifically used to:

[0014] Performing feature mixing processing on the target EEG feature and the target EMG feature to obtain mixed EEG feature;

[0015] The brain-muscle features are input into a plurality of behavior detection models to obtain corresponding behavior probability values, respectively, and each behavior detection model corresponds to a preset driving behavior.

[0016] Optionally, when the preset hybrid strategy is a classified hybrid strategy, the detection unit is specifically configured to:

[0017] Inputting the target EEG feature and the target EMG feature into an EEG classifier and an EMG classifier, respectively, to obtain an EEG classification set and an EMG classification set; the EEG classification set and the EMG classification set both contain prediction probability values ​​corresponding to each preset driving behavior;

[0018] Based on a preset hybrid sub-strategy, the results of the EEG classification set and the EMG classification set are judged to obtain a plurality of behavior probability values; each behavior probability value corresponds to a preset driving behavior.

[0019] Optionally, when the preset hybrid strategy is a classified hybrid strategy, the detection unit is specifically configured to:

[0020] For each preset driving behavior, the predicted probability values ​​in the EEG classification set and the EMG classification set are calculated to obtain the behavior probability value of each preset driving behavior; or,

[0021] For each preset driving behavior, a behavior probability value of each preset driving behavior is obtained by performing or calculating the predicted probability values ​​in the EEG classification set and the EMG classification set.

[0022] Optionally, when the preset hybrid strategy is a Bayesian hybrid strategy, the detection unit is specifically used to:

[0023] Based on the prior probability of each preset driving behavior, the conditional probability of the target EEG feature and the target EMG feature is calculated; the prior probability represents the probability of each preset driving behavior occurring when there is no EEG signal and EMG signal; the conditional probability represents the probability of each EEG signal and EMG signal occurring under the corresponding preset driving behavior;

[0024] Based on the obtained conditional probabilities, the posterior probability of each preset driving behavior is calculated; the posterior probability represents the probability of each preset driving behavior occurring under any combination of EEG signals and EMG signals;

[0025] A maximum a posteriori probability value is determined from the obtained multiple posterior probabilities, and the maximum a posteriori probability value is used as the behavior probability value of the corresponding preset driving behavior.

[0026] Optionally, the control unit is specifically used to:

[0027] Determine a maximum behavior probability value from the multiple behavior probability values, and determine a preset driving behavior corresponding to the maximum behavior probability value as the target driving behavior; or,

[0028] Based on the decision tree classification strategy, the multiple behavior probability values ​​are judged step by step to determine the target driving behavior.

[0029] Optionally, the multiple behavior probability values ​​include an emergency braking probability value and a soft braking probability value, and the control unit is specifically used to:

[0030] Based on the emergency braking probability value and a first preset threshold, determining whether the target driving behavior is an emergency braking behavior;

[0031] If the target driving behavior is not the emergency braking behavior, whether the target driving behavior is a soft braking behavior is determined based on the soft braking probability value and a second preset threshold.

[0032] In a third aspect, the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any one of the vehicle control methods based on hybrid brain-muscle signals described in the first aspect is implemented.

[0033] In a fourth aspect, the present application provides a computer storage medium, wherein the computer-readable storage medium stores computer program instructions, and the computer program instructions are executed by a processor to implement any one of the vehicle control methods based on hybrid brain-muscle signals in the above-mentioned first aspect.

[0034] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising computer program instructions, which, when executed by a processor, implement any one of the vehicle control methods based on hybrid brain-muscle signals in the first aspect.

[0035] The beneficial effects of the present invention are as follows:

[0036] The embodiment of the present application provides a vehicle control method based on a mixed brain-muscle signal, which obtains a mixed brain-muscle signal set consisting of multiple EEG signals and multiple electromyographic signals of a target object, and performs feature processing on the mixed brain-muscle signal set to obtain target EEG features and target electromyographic features. Through a preset hybrid strategy, driving behavior detection is performed on the target EEG features and target electromyographic features, and the behavior probability values ​​corresponding to each of the multiple preset driving behaviors are obtained, thereby determining the target driving behavior of the target object through multiple behavior probability values, and controlling the vehicle accordingly. In this way, the signal advantages of the EEG signals and electromyographic signals are combined to more comprehensively analyze and predict the driving intention of the target object to improve the accuracy of driving behavior prediction and the reliability and safety of vehicle control. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0038] Figure 1 A schematic flow chart of a vehicle control method based on mixed brain-muscle signals provided in an embodiment of the present application;

[0039] Figure 2 A schematic diagram of a signal collection position provided in an embodiment of the present application;

[0040] Figure 3 A schematic diagram of an electromyographic signal collection position provided in an embodiment of the present application;

[0041] Figure 4 A schematic diagram of a brain-muscle signal processing process provided in an embodiment of the present application;

[0042] Figure 5 A schematic diagram of the structure of a vehicle control device based on mixed brain-muscle signals provided in an embodiment of the present application;

[0043] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be arbitrarily combined with each other. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0045] The terms "first" and "second" in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive protection. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. "Multiple" in the present application can mean at least two, for example, can be two, three or more, and the embodiments of the present application are not limited.

[0046] The term "and / or" in the embodiments of the present application is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0047] The following is a brief introduction to the design concept of the embodiments of the present application.

[0048] With the rapid development of automobile technology, traffic accidents still cause safety and economic losses frequently. ADAS came into being, which can assist drivers through various means, especially playing an important role in detecting driver status and driving behavior, and can effectively reduce the incidence of traffic accidents. In the prior art, traditional driving behavior detection systems mainly rely on signals such as vehicle information, environmental information and driver behavior information, but the acquisition of these signals is limited, and it is difficult to accurately predict the driver's intentions, resulting in low efficiency in driving behavior detection and low reliability and safety of vehicle control.

[0049] Therefore, brain-computer interface (BCI) technology has shown great potential in the field of driving behavior detection by virtue of its ability to decode brain signals. However, when predicting the driver's braking intention through electroencephalography (EEG), it is difficult to apply it to complex actual driving environments due to the limitations of EEG's stability and reliability. At the same time, the detection system with a single signal source is susceptible to interference and it is difficult to provide comprehensive and reliable behavior prediction.

[0050] To solve the above problems, the embodiment of the present application provides a vehicle control method based on a mixed brain-muscle signal, which obtains a mixed brain-muscle signal set consisting of multiple EEG signals and multiple electromyographic signals of the target object, and performs feature processing on the mixed brain-muscle signal set to obtain target EEG features and target electromyographic features. Through a preset hybrid strategy, the target EEG features and target electromyographic features are detected for driving behavior, and the behavior probability values ​​corresponding to multiple preset driving behaviors are obtained, so that the target driving behavior of the target object is determined through multiple behavior probability values, and the vehicle is controlled accordingly. In this way, the driving intention of the target object is more comprehensively analyzed and predicted by combining the signal advantages of the EEG signals and electromyographic signals, so as to improve the accuracy of driving behavior prediction and the reliability and safety of vehicle control. In addition, the present application adopts a preset hybrid strategy to accurately calculate the probability of occurrence of each driving behavior, and can identify different driving behaviors according to the driver's EEG activity and muscle reaction, so as to better adapt to different driving situations and ensure accurate identification of the driver's behavioral intention.

[0051] The following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In the specific implementation process, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0052] The solution provided by the embodiment of the present application can be applied to most vehicle driving application scenarios, and is used to improve the accuracy of driving behavior detection and the safety and reliability of vehicle driving. For example, the driver will encounter different traffic conditions during driving, and usually take driving behaviors such as soft braking and emergency braking according to the specific traffic conditions. Taking soft braking as an example, when the driver encounters a situation that may lead to an emergency or sees a prompt message during driving, the driver will actively perform soft braking to avoid the occurrence of an emergency. For example, pedestrians gather on the front side, and the driver performs soft braking to slow down and avoid pedestrians suddenly crossing the road to cause an emergency. Or, the driver is driving at a high speed, and in order to ensure safe driving, he actively performs a deceleration operation. Or, the front sign prompts "School ahead, slow down", the driver obeys the traffic prompts, performs soft braking, and avoids emergencies. Taking emergency braking as an example, when the driver is driving, facing emergencies, the driver passively takes emergency braking behavior to avoid traffic accidents. For example, in the case of close following, the front vehicle suddenly slows down, the front vehicle's brake lights light up, and emergency braking is taken. Or, a close adjacent vehicle suddenly inserts, and emergency braking is taken. Or, a pedestrian at a close distance suddenly crosses the road, and emergency braking is performed. The vehicle control method based on hybrid electromyographic signals provided in the embodiment of the present application can collect EEG signals and electromyographic signals of target objects such as drivers in real time, detect the driver's driving behaviors such as emergency braking and soft braking, and transmit the detected driving behaviors to the vehicle control system, so as to control the vehicle in time to directly perform driving behaviors such as emergency braking or soft braking, thereby improving the reliability and safety of vehicle driving.

[0053] Of course, the method provided in the embodiment of the present application is not limited to the above application scenarios, but can also be used in other possible application scenarios, which are not limited by the embodiment of the present application. The functions that can be realized by each device in the above application scenarios will be described in the subsequent method embodiments, and no further details will be given here.

[0054] Below, in combination with the application scenarios described above, the method provided by the exemplary embodiment of the present application is described with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the implementation methods of the present application are not limited in this regard.

[0055] like Figure 1 As shown, the embodiment of the present application provides a vehicle control method based on hybrid electromyographic signals, and the specific process of the method is as follows:

[0056] Step 101: Acquire a mixed brain-muscle signal set of a target object, and perform feature processing on the mixed brain-muscle signal set to obtain target EEG features and target EMG features.

[0057] In the embodiments of the present application, the target object refers to an individual related to driving behavior, that is, an individual who is responsible for operating the vehicle and generating EEG signals and EMG signals. For example, the most common target object is the driver who performs vehicle driving operations, or in other specific scenarios, monitoring people other than the driver may also be of reference significance. In this way, the present application collects the EEG activity and muscle electrical activity of the target object to determine its corresponding driving behavior, thereby accurately judging its behavioral intention and realizing intelligent assisted driving.

[0058] Specifically, in order to overcome the limitations of traditional BCI technology, this application adopts hybrid brain-computer interface (hBCI) technology, that is, by combining multiple signal sources such as EEG signals, electromyographic signals, electrocardiographic signals, environmental information and vehicle information, to improve the detection accuracy and reliability of driving behavior. Among them, EEG signals are signals that reflect the activity of neurons in the cerebral cortex and provide important information about the state of the brain, such as consciousness state, emotional changes, cognitive load, etc., which can be used to judge the driver's attention, fatigue level, emotional state or behavioral intention, etc. Electromyography (EMG) is a signal that reflects the electrical activity of muscles and reveals information such as the contraction state, fatigue level, and movement intention of muscles. In driving behavior detection, electromyography can be used to understand the driver's limb movements, muscle load and movement intention. For example, during driving, when the driver brakes or accelerates, the muscles of the legs or arms will generate electromyography. For example, when emergency braking is performed, the electromyography generated by muscle contraction will reflect the driver's strong reaction to the brake pedal. Alternatively, if the driver is in a normal driving state for a long time, there may be slight muscle relaxation or continuous weak muscle activity, and these electromyographic signals can be used to determine whether the driver has entered a fatigue state. Therefore, electromyographic signals, as a signal directly related to muscle movement, have good reliability and predictability. The embodiment of the present application fuses electromyographic signals with electroencephalogram signals, reflects the driver's brain activity state through electroencephalogram signals, and captures the driver's muscle activity through electromyographic signals, so as to more comprehensively analyze the driver's state and behavior, and more accurately and reliably predict the driver's driving behavior intention, thereby providing more accurate information for ADAS, improving the driver's safety prevention capabilities, and effectively reducing the incidence of traffic accidents.

[0059] In a possible implementation, the embodiment of the present application may pre-set multiple preset driving behaviors to be determined based on actual driving behavior detection and assistance needs, thereby determining the intended driving behavior of the target object through the EEG signals and EMG signals of the target object.

[0060] Specifically, the preset driving behaviors may include emergency braking, soft braking, and normal driving. Emergency braking behavior refers to the rapid braking behavior performed by the driver in an emergency situation, which is usually manifested as a fast and strong braking operation. At this time, the driver may feel a strong emergency reaction (for example, to avoid a collision). The embodiment of the present application can capture the driver's sharp reaction from the EEG signal and the EMG signal, such as the sudden contraction of the leg muscles and the tension signal of the brain. Soft braking behavior refers to the braking operation performed by the driver under relatively stable conditions. Soft braking is usually used for slow deceleration, which is manifested as a lighter reaction and a smaller brain wave amplitude in the brain and muscle signals, as well as a slow contraction of the muscles. Normal driving behavior refers to the behavior performed by the driver during normal driving, without involving any emergency or special circumstances. At this time, the driver's EEG signal and EMG signal reflect a relatively stable and conventional driving mode.

[0061] In a possible implementation, the embodiment of the present application can collect multiple EEG signals and multiple EMG signals of the target object by presetting the signal collection positions of the EEG signals and the EMG signals.

[0062] Specifically, the embodiments of the present application can pre-set each signal collection position according to the position and distribution rules in the international standard 10 / 20 system. The international standard 10 / 20 system is a standardized method for placing EEG electrodes, which aims to ensure the comparability of electrodes between different individuals. The placement of its electrodes is based on the anatomical landmarks of the head. By measuring the size of the head, the electrodes are evenly distributed at a ratio of 10% or 20% of the head circumference. For example, according to the distribution of head circumference, the head is divided into frontal (Frontal), parietal (P), occipital (O), central (C), and temporal (Temporal) regions. Letters represent brain regions, such as F (frontal), P (parietal), O (occipital), C (central), and T (temporal). Numbers represent positions, odd numbers (1, 3, 5, 7) are on the left, even numbers (2, 4, 6, 8) are on the right, and z represents the center (Zero Line). Reference Figure 2The figure shows a schematic diagram of a signal acquisition position provided by an embodiment of the present application. According to the position and distribution rules, the embodiment of the present application will select F3, F4, Fz, Cz, Pz, C3, C4, T7, T8, P7, P3, P4, P8, O1, 02 and Oz channels as the signal acquisition positions of the EEG signal, and use the potential average of the two acquisition positions A11 and A12 as the reference potential. A11 and A12 represent the electrodes of the left and right earlobes, which are stable in position and less subject to interference. Using them as reference potentials and providing a symmetrical average benchmark with their average values ​​can reduce the error caused by electrode benchmark offset. The electrode points selected for the entire EEG signal cover the frontal lobe, central lobe, parietal lobe, occipital lobe and temporal lobe, which are the core areas for controlling movement, vision, sensation and other functions, and can ensure that the EEG signal acquisition can fully reflect the brain activities related to driving behaviors such as emergency braking and soft braking during driving.

[0063] Specifically, refer to Figure 3 The figure shows a schematic diagram of an electromyographic signal collection position provided by an embodiment of the present application. In the embodiment of the present application, the rectus femoris, vastus medialis, vastus lateralis, tibialis anterior, biceps femoris, medial gastrocnemius, lateral gastrocnemius and soleus can be selected as the signal collection position of the electromyographic signal. Among them, the rectus femoris, vastus medialis and vastus lateralis are mainly used to control the extension of the knee joint. The rectus femoris is a major muscle in the front of the thigh, which participates in actions such as standing, walking and running. The vastus medialis is mainly used for the extension of the knee joint and helps control the internal rotation of the knee joint, especially playing an important role when walking and running. The vastus lateralis is mainly used for the extension of the knee joint and participates in the external rotation of the knee joint, helping to stabilize the knee and promote the abduction of the calf. The tibialis anterior controls the dorsiflexion and inversion of the ankle, the biceps femoris controls the flexion of the knee joint, the medial gastrocnemius and the lateral gastrocnemius participate in the main actions of the calf, control the thrust of the foot, and the soleus is responsible for the plantar flexion of the ankle. By collecting the muscle signals that are directly involved in the braking operation during driving, the muscle activities of soft braking and emergency braking can be accurately reflected. The changes in the corresponding electromyographic signals can characterize the driver's action intentions and provide accurate auxiliary judgment basis.

[0064] In a possible implementation, the embodiment of the present application may use a 32-channel Xintuo acquisition device to collect EEG signals and EMG signals. As a high-precision signal acquisition device, the 32-channel Xintuo acquisition device has 32 independent acquisition channels, which can simultaneously record bioelectric signals from multiple points, and supports a sampling frequency of up to 1000Hz, ensuring high time domain resolution and accuracy of signal acquisition. In this way, a sufficient number of channels can be provided by using a 32-channel Xintuo acquisition device, covering 16 channels of EEG signals and 8 channels of EMG signals. Ensure the comprehensiveness and precision of signal acquisition. The sampling frequency of 1000Hz is used to capture subtle changes in brain and muscle activity, which is more suitable for short-term responses such as emergency braking or soft braking, and the device can provide a real-time data acquisition interface, laying the foundation for subsequent preprocessing and feature extraction.

[0065] In a possible implementation, before the mixed brain muscle signal set is feature processed, the embodiment of the present application will also pre-process the EEG signal and the EMG signal, including: In order to reduce noise interference, the embodiment of the present application can use filtering technology to remove unnecessary frequency bands of the EEG signal and the EMG signal, such as low-frequency noise and high-frequency interference. In order to reduce the amount of calculation and improve processing efficiency, the embodiment of the present application can downsample the sampling frequency of the signal to ensure that the signal is sufficiently accurate while avoiding redundancy. In order to further improve the signal quality, the embodiment of the present application can use independent component analysis to effectively separate the signal source and extract the independent components in the EEG and EMG signals, thereby removing baseline drift (for example, slow-frequency noise caused by poor electrode contact) and common noise (for example, electrooculographic artifacts, electromyographic artifacts and other physiological artifacts), significantly improving the quality of the signal, and providing more accurate data support for subsequent feature extraction and behavior classification.

[0066] Specifically, for EEG signals, the embodiment of the present application can use a sixth-order Butterworth bandpass filter to filter the EEG signals to obtain a target frequency band of 0.5Hz to 60Hz. Further, in order to reduce the calculation time as much as possible and improve the processing efficiency of EEG signals, the embodiment of the present application can use a downsampling method to reduce the sampling frequency of the EEG signals to 200Hz. Then, independent component analysis is used to filter out the electrooculographic artifacts and electromyographic artifacts that may exist in the EEG signals, and baseline correction and common average reference are used to eliminate the baseline drift and common noise of the EEG signals. For electromyographic signals, the embodiment of the present application can use a sixth-order Butterworth bandpass filter to filter the electromyographic signals to obtain a target frequency band of 15Hz to 90Hz. In order to keep pace with the sampling frequency of the EEG signals, the sampling frequency of the electromyographic signals will be reduced to 200Hz by downsampling. Then, baseline correction and common average reference are used to eliminate the baseline drift and common noise in the electromyographic signals.

[0067] In a possible implementation, the embodiment of the present application can perform signal projection and feature calculation on the EEG and EMG in the mixed brain muscle signal set to obtain the original frequency domain features of the EEG and EMG. According to the correlation between each original frequency domain feature and the preset driving behavior, the features with higher correlation are screened out from each original frequency domain feature as the target EEG feature and the target EMG feature.

[0068] Specifically, since the number of channels of the original EEG and EMG signals is large, direct use may lead to data redundancy and increased computational complexity. The embodiment of the present application will adopt a common spatial pattern (Common Spatial Pattern, CSP) to project 16-channel EEG signals and 8-channel EMG signals into m-channel EEG signals and M-channel EMG signals. Among them, the m and M values ​​are usually less than the original number of channels and can be set according to task requirements and classification accuracy, for example, 2 to 6 channels. The m and M values ​​represent the optimal feature channel numbers of the EEG and EMG signals extracted after CSP projection, respectively. In this way, m and M projection channels are extracted, the feature dimension is reduced, the feature calculation efficiency is improved, and the information highly related to the classification task can be retained to ensure the accuracy of driving behavior detection.

[0069] Next, the embodiment of the present application can perform a fast Fourier transform on the projected EEG and EMG, and calculate the power spectral density (PSD) of each EEG and EMG as the original frequency domain features of each EEG and EMG. And use distance correlation analysis (DCC) to calculate the distance correlation coefficients between the original frequency domain features of EEG and EMG and the target category (for example, different types of preset driving behaviors), so as to select multiple features with correlations higher than a preset threshold from the original frequency domain features, that is, the features most relevant to the target category, as the target EEG features and target EMG features.

[0070] Step 102: Based on a plurality of preset behavior detection models, driving behavior detection is performed on the target EEG features and the target EMG features to obtain a plurality of behavior probability values.

[0071] In the embodiment of the present application, the behavior detection model is mainly used to determine the target driving behavior of the target object through the target EEG characteristics and target EMG characteristics of the target object, and generate a corresponding behavior probability value. The behavior probability value represents the possibility of different driving behaviors reflected by the EEG signal and EMG signal of the target object, and is also the prediction confidence of the behavior detection model for a specific driving behavior.

[0072] Specifically, the present application may pre-set the corresponding preset driving behavior classification for each behavior detection model to distinguish between target driving behavior and non-target driving behavior. For example, emergency braking vs. non-emergency braking, this classification is used to determine whether the driver has performed emergency braking. Emergency braking behavior usually has a strong EEG response (such as higher frequency brain waves, which may reflect the driver's tension and stress state) and obvious electromyographic response (such as rapid contraction of leg muscles). Non-emergency braking is manifested as relatively gentle EEG signals and electromyographic signals. Soft braking vs. non-soft braking, this classification is used to identify whether the driver has performed a soft braking operation. During soft braking, the EEG signal may show a relatively stable response, and the electromyographic signal is manifested as a lower muscle contraction amplitude. Non-soft braking may be manifested as higher or lower EEG or electromyographic activity, or as other activity patterns unrelated to braking. Normal driving vs. abnormal driving, this classification is used to determine whether the driver is in a normal driving state. During normal driving, the EEG signal is usually stable, and the electromyographic signal is relatively weak, and does not show obvious stress or urgent reactions. Abnormal driving behavior (such as abnormal behavior caused by driver fatigue, lack of concentration, etc.) will cause abnormal changes in EEG and EMG signals. In this way, the behavior detection model can judge the driver's current driving behavior based on multiple preset driving behavior classification standards by analyzing the features extracted from EEG and EMG signals, and assign corresponding probability values ​​to the driving behavior, reflecting the model's confidence in the driving behavior.

[0073] In a possible implementation, the embodiment of the present application may adopt a regularized linear discriminant analysis (RLDA) classification model as a behavior detection model.

[0074] Specifically, the expression of the RLDA classification model is as follows:

[0075] f rlda (x) = w T x

[0076]

[0077]

[0078] Where x represents the feature vector of EEG signal and EMG signal, w represents the feature vector projection matrix, and frlda represents the final classification decision result, which is used to determine the category of the preset driving behavior indicated by the input feature x, such as emergency braking, soft braking or normal driving.

[0079] u1 and u2 represent the average values ​​of the fused feature vectors of two samples of different classes (e.g., emergency braking and non-emergency braking), respectively.

[0080] ∑w represents the intra-class dispersion matrix, which is used to measure the distribution of samples of the same category. It reflects the degree of variation of samples of each category and plays a key role in the calculation of the weight vector. d represents the dimension of ∑w.

[0081] λ represents the regularization parameter, which is used to balance the complexity and fitting ability of the model. Regularization can avoid overfitting of the model and ensure that it has good generalization ability for new data.

[0082] trace() represents the adjustment part of the intra-class scatter matrix, which controls the expansion of the feature space by tracking the matrix to ensure the maximization of the distance between categories.

[0083] I is the identity matrix, which is used for regularization calculation so that all feature dimensions are subject to the same constraints, thereby maintaining the stability of the model.

[0084] In a possible implementation, the embodiment of the present application may adopt a parallel hybrid strategy or a serial hybrid strategy to detect the target EEG features and the target EMG features for driving behavior, so as to obtain the processing process of the behavior probability value of each preset driving behavior. The specific implementation process of the parallel hybrid strategy and the serial hybrid strategy will be explained in detail below:

[0085] Specifically, the parallel hybrid strategy includes the feature hybrid strategy (hBCI-FL) and the classifier hybrid strategy (hBCI-CL), and the classifier-level hybrid includes the mixed strategy (hBCI-CL1), the or mixed strategy (hBCI-CL2) and the Bayesian hybrid strategy (hBCI-CL3).

[0086] In a possible implementation, when the preset mixing strategy is a feature mixing strategy, that is, a mixing at the feature level, the embodiment of the present application performs feature mixing processing on the target EEG features and the target EMG features to obtain mixed brain-muscle features. Then, the brain-muscle features are respectively input into the behavior detection models corresponding to the various preset driving behaviors to obtain the corresponding behavior probability values.

[0087] Specifically, the embodiment of the present application can perform data normalization processing on the target EEG features and the target EMG features respectively, and splice the standardized target EEG features and the target EMG features to form a joint feature matrix, which are respectively input into the RLDA classification model corresponding to the three preset driving behaviors to obtain the classification results, i.e., the probability values ​​corresponding to the three preset driving behaviors of emergency braking, soft braking, and normal driving.

[0088] In a possible implementation, when the preset hybrid strategy is a classification hybrid strategy, that is, a hybrid at the classifier level, the embodiment of the present application will obtain an EEG classification set and an EMG classification set containing the prediction probability values ​​corresponding to each preset driving behavior by inputting the target EEG features and the target EMG features into the EEG classifier and the EMG classifier, respectively. Thus, based on the preset hybrid sub-strategy, the EEG classification set and the EMG classification set are judged, and the behavior probability values ​​corresponding to the preset driving behaviors are obtained.

[0089] In a possible implementation, when the preset mixed strategy is an AND strategy in the classification mixed strategy, the embodiment of the present application can perform AND calculations for each preset driving behavior based on the predicted probability values ​​in the EEG classification set and the EMG classification set, so as to obtain the behavior probability value of each preset driving behavior.

[0090] Specifically, in the hybrid strategy (hBCI-CL1), the target EEG features and target EMG features can be input into the corresponding RLDA classification model to obtain the probability values ​​of various preset driving behaviors output by the EEG classifier and the EMG classifier. Then, for each preset driving behavior, the corresponding probability values ​​output by the EEG classifier and the EMG classifier are multiplied to obtain the probability product corresponding to each preset driving behavior as the behavior probability value of the preset driving behavior.

[0091] In a possible implementation, when the preset mixed strategy is an OR strategy in the classification mixed strategy, the embodiment of the present application can also perform OR calculation for each preset driving behavior based on the predicted probability values ​​in the EEG classification set and the EMG classification set to obtain the behavior probability value of each preset driving behavior.

[0092] Specifically, in the hybrid strategy (hBCI-CL2), the target EEG features and target EMG features can be respectively input into the corresponding RLDA classification model to obtain the probability values ​​of various preset driving behaviors output by the EEG classifier and the EMG classifier. Then, for each preset driving behavior, a relatively large probability value is selected from the probability values ​​output by the EEG classifier and the EMG classifier as the behavior probability value of the preset driving behavior.

[0093] In a possible implementation, the mixing at the classifier level also includes Bayesian mixing, that is, when the preset mixing strategy is a Bayesian mixing strategy, the embodiment of the present application will calculate the conditional probabilities of the target EEG features and the target EMG features according to the probability of each preset driving behavior occurring when there are no EEG signals and EMG signals, that is, the prior probability of each preset driving behavior, so as to determine the probability of each EEG signal and EMG signal occurring under the corresponding preset driving behavior. Then, the posterior probability of each preset driving behavior is calculated by obtaining each conditional probability, that is, the probability of each preset driving behavior occurring under any combination of EEG signals and EMG signals. Thus, from the multiple posterior probabilities obtained, the maximum posterior probability value is determined as the behavior probability value of the corresponding preset driving behavior.

[0094] Specifically, in the Bayesian hybrid strategy (hBCI-CL3), the prior probability model P(R|E1,E2) of the mixed EEG and EMG can be set first. This model describes how to calculate the posterior probability of the preset driving behavior R based on the target EEG feature E1 and the target EMG feature E2 through the Bayesian theorem.

[0095] By using Bayes' theorem, the prior probability model can be decomposed into the following expression:

[0096] P(R|E1,E2)∝P(R)P(E1,E2|R)

[0097] Among them, R represents the classification category, i.e., the corresponding preset driving behavior;

[0098] E1 and E2 represent the classification results of the EEG classifier and the EMG classifier, respectively.

[0099] P(R) represents the prior probability, that is, the probability of the preset driving behavior occurring in the absence of EEG and EMG signals.

[0100] P(E1,E2|R) represents the joint conditional probability of EEG and EMG occurring simultaneously under the preset driving behavior R.

[0101] Assuming that the EEG and EMG signal sources are independent of each other, the following expression can be obtained:

[0102] P(E1,E2|R)=P(E1|R)P(E2|R)

[0103] R out =argmax(P(R=h)P(E1|R=h)P(E2|R=h))

[0104] Among them, h∈(target, non-target) represents

[0105] P(E1|R=h) and P(E2|R=h) are the conditional probabilities of EEG and EMG under the preset driving behavior R, respectively, where P(R=h) is the same for both signals.

[0106] R out The behavior probability value of the preset driving behavior that represents the final output is the final result after Bayesian calculation and maximization of the posterior probability. The system will output the maximum posterior probability as the behavior probability value of the preset driving behavior.

[0107] In a possible implementation, the serial hybrid strategy (hBCI-SE) refers to decomposing the behavior detection task of multiple preset driving behaviors into two parts, and using EEG signals and EMG signals for detection and processing respectively.

[0108] Specifically, considering the importance of emergency braking, the embodiment of the present application may treat the emergency braking detection task as an independent part, and the soft braking and normal driving detection tasks as other parts, so as to realize driving behavior detection by adopting two methods: EEG signal priority processing and EMG signal priority processing. For example, with the EEG-prioritized serial hybrid strategy (hBCI-SE1), the EEG signal is used alone to detect emergency braking and non-emergency braking behaviors, while the EMG signal is used to detect soft braking and non-soft braking behaviors, normal driving and abnormal driving behaviors. In the EMG-prioritized serial hybrid strategy (hBCI-SE2), the EMG signal is used to detect emergency braking and non-emergency braking tasks, and the EEG signal is used to detect soft braking and non-soft braking behaviors, normal driving and abnormal driving behaviors.

[0109] In one possible implementation, reference Figure 4 FIG. 1 is a schematic diagram of a brain muscle signal processing process provided by an embodiment of the present application. Figure 4In the embodiment of the present application, 16 channels of EEG signals are obtained from the driver's EEG signal acquisition device, and 8 channels of EMG signals are obtained from the driver's muscle activity sensor (such as legs) to capture the driver's muscle reaction (such as braking). And filtering, downsampling, baseline correction, common average reference and other preprocessing are performed on the EEG signal and the EMG signal respectively to remove signal noise, extract effective features, and ensure the accuracy of subsequent analysis. In particular, independent component analysis will be performed on the EEG signal to remove artifacts in the EEG signal. Then, feature calculation and feature selection are performed on the EEG signal and the EMG signal respectively, for example, the signal is converted into frequency domain features (such as power spectral density) by methods such as Fourier transform, and the target frequency domain features that are most helpful for classification are selected. According to the multiple probability values ​​obtained, the driving behavior corresponding to the maximum probability value is finally selected. After obtaining each target EEG feature and each target EMG feature, a feature mixing strategy, a classifier mixing strategy or a serial mixing strategy will be adopted to detect driving behavior on each target EEG feature and each target EMG feature, and obtain the behavior probability value of each preset driving behavior. Among them, the classifier hybrid strategy is to input the target EEG features and target EMG features into the RLDA classification model respectively, obtain the probability value P(x) output by the EEG classifier and the EMG classifier respectively, and calculate the behavior probability value of each preset driving behavior according to the EEG signal weight w1 and the EMG signal weight w2.

[0110] Step 103: Determine a target driving behavior of the target object based on the multiple behavior probability values.

[0111] In the embodiment of the present application, after obtaining the behavior probability value corresponding to each preset driving behavior, the target driving behavior of the target object is determined from multiple preset driving behaviors through the preset one-to-many classification strategy and decision tree classification strategy.

[0112] Specifically, the embodiment of the present application will determine the actual operating intention of the target object, that is, the target driving behavior, from multiple preset driving behaviors through a one-to-many classification strategy and a decision tree classification strategy for the behavior probability value of each preset driving behavior obtained through a serial or parallel hybrid strategy in the aforementioned process.

[0113] In a possible implementation, the embodiment of the present application is based on a one-to-many classification strategy, and can determine a maximum behavior probability value from multiple behavior probability values, and determine a preset driving behavior corresponding to the maximum behavior probability value as a target driving behavior.

[0114] Specifically, taking the three-classification task of three preset driving behaviors, namely emergency braking, soft braking and normal driving, as an example, the embodiment of the present application can decompose the three-classification task into three parallel two-classification tasks through a one-to-many classification strategy, including emergency vs. non-emergency braking, soft braking vs. non-soft braking, and normal driving vs. abnormal driving. And compare the preset thresholds of the RLDA classification model corresponding to the three preset driving behaviors with the behavior probability values ​​corresponding to the three preset driving behaviors to determine the corresponding threshold relative values, that is, the degree of deviation of the behavior probability value from the preset threshold. According to the preset decision rules, the behavior probability value with the largest threshold relative value is determined as the maximum behavior probability value, and its corresponding preset driving behavior is determined as the target driving behavior. Among them, the expression of the decision rule can be as follows:

[0115]

[0116] Among them, R a is the output result of the one-to-many classification strategy, that is, the maximum behavior probability value.

[0117] y i is the relative value of the threshold output by the i-th RLDA classification model. In this formula, three parallel binary classification tasks are taken as an example, i = {1, 2, 3}.

[0118] In a possible implementation, the embodiment of the present application may determine the target driving behavior by performing step-by-step judgment on multiple behavior probability values ​​according to a decision tree classification strategy.

[0119] In a possible implementation, the preset driving behaviors may include emergency braking, soft braking, and normal driving, and the corresponding multiple behavior probability values ​​include an emergency braking probability value and a soft braking probability value. In the embodiment of the present application, a first preset threshold is set for the emergency braking behavior, and a second preset threshold is set for the soft braking behavior. By comparing the emergency braking probability value with the first preset threshold, it can be determined whether the target driving behavior is an emergency braking behavior. If the target driving behavior is not an emergency braking behavior, the soft braking probability value is compared with the second preset threshold to determine whether the target driving behavior is a soft braking behavior.

[0120] Specifically, taking the three-classification task of emergency braking, soft braking and normal driving as an example, the decision tree classification strategy in the embodiment of the present application decomposes the three-classification task into two progressive two-classification tasks, namely emergency braking vs non-emergency braking, soft braking vs normal driving, and sets corresponding preset thresholds for the RLDA classification models corresponding to these two two-classification tasks. According to the decision rule, the behavior probability values ​​corresponding to the two two-classification tasks are compared with the preset thresholds of the corresponding RLDA classification models in turn. For example, first compare the size between the behavior probability value y1 corresponding to the emergency braking behavior and the preset threshold K1 of the corresponding classification model. When y1 is greater than K1, the emergency braking behavior is determined as the target driving behavior, and no subsequent judgment is performed. Only when y1 is less than K1, continue to compare the size between the behavior probability value y2 corresponding to the soft braking behavior and the preset threshold K2 of the corresponding classification model. When y2 is greater than K2, the soft braking behavior is determined as the target driving behavior, otherwise the normal driving behavior is determined as the target driving behavior.

[0121] Step 104: Based on the target driving behavior, the vehicle is controlled accordingly.

[0122] In an embodiment of the present application, after the target driving behavior of the target object is determined through EEG signals and EMG signals, the target driving behavior intention can be transmitted to the relevant control system of the driving vehicle so as to control the vehicle to complete the detected target driving behavior, such as emergency braking and soft braking, thereby realizing human-computer interactive collaborative operation, giving full play to the advantages of EEG signals and EMG signals, and improving the accuracy of driving behavior detection and the safety and reliability of vehicle driving.

[0123] In a possible implementation, the embodiment of the present application can decompose the data of EEG signals and EMG signals into a training data set and a test data set through six-fold cross validation. The training data set is used to train various behavior detection models such as RLDA classification models, and the test data set is used for model testing. After the model is tested, the driver's emergency braking, soft braking and other driving behaviors can be detected based on the EEG signals and EMG signals currently collected in real time, and the detected target driving behavior intentions can be transmitted to the vehicle control system to control the vehicle to directly perform the relevant driving behaviors.

[0124] Based on the same inventive concept, the embodiment of the present application further provides a vehicle control device 50 based on a hybrid electromyographic signal, the device comprising:

[0125] The acquisition unit 501 is used to acquire a mixed brain-muscle signal set of a target object, and perform feature processing on the mixed brain-muscle signal set to obtain a target EEG feature and a target EMG feature; the mixed brain-muscle signal set includes multiple EEG signals and multiple EMG signals of the target object;

[0126] The detection unit 502 is used to perform driving behavior detection on the target EEG feature and the target EMG feature based on a preset hybrid strategy to obtain a plurality of behavior probability values; each behavior probability value corresponds to a preset driving behavior;

[0127] The control unit 503 is used to determine the target driving behavior of the target object based on the multiple behavior probability values, and to control the vehicle accordingly based on the target driving behavior.

[0128] Optionally, when the preset mixing strategy is a feature mixing strategy, the detection unit 502 is specifically configured to:

[0129] Performing feature mixing processing on the target EEG feature and the target EMG feature to obtain mixed EEG feature;

[0130] The brain-muscle features are input into multiple behavior detection models to obtain corresponding behavior probability values. Each behavior detection model corresponds to a preset driving behavior.

[0131] Optionally, when the preset hybrid strategy is a classified hybrid strategy, the detection unit 502 is specifically configured to:

[0132] Inputting the target EEG features and the target EMG features into the EEG classifier and the EMG classifier, respectively, to obtain an EEG classification set and an EMG classification set; the EEG classification set and the EMG classification set both contain prediction probability values ​​corresponding to each preset driving behavior;

[0133] Based on the preset hybrid sub-strategy, the results of the EEG classification set and the EMG classification set are judged to obtain multiple behavior probability values; each behavior probability value corresponds to a preset driving behavior.

[0134] Optionally, when the preset hybrid strategy is a classified hybrid strategy, the detection unit 502 is specifically configured to:

[0135] For each preset driving behavior, the predicted probability values ​​in the EEG classification set and the EMG classification set are calculated to obtain the behavior probability value of each preset driving behavior; or,

[0136] For each preset driving behavior, a behavior probability value of each preset driving behavior is obtained by performing or calculating the predicted probability values ​​in the EEG classification set and the EMG classification set.

[0137] Optionally, when the preset hybrid strategy is the Bayesian hybrid strategy, the detection unit 502 is specifically configured to:

[0138] Based on the prior probability of each preset driving behavior, the conditional probability of the target EEG feature and the target EMG feature is calculated; the prior probability represents the probability of each preset driving behavior occurring when there is no EEG signal and EMG signal; the conditional probability represents the probability of each EEG signal and EMG signal occurring under the corresponding preset driving behavior;

[0139] Based on the obtained conditional probabilities, the posterior probability of each preset driving behavior is calculated; the posterior probability represents the probability of each preset driving behavior occurring under any combination of EEG signals and EMG signals;

[0140] A maximum a posteriori probability value is determined from the obtained multiple posterior probabilities, and the maximum a posteriori probability value is used as the behavior probability value of the corresponding preset driving behavior.

[0141] Optionally, the control unit 503 is specifically configured to:

[0142] Determine a maximum behavior probability value from the multiple behavior probability values, and determine the preset driving behavior corresponding to the maximum behavior probability value as the target driving behavior; or,

[0143] Based on the decision tree classification strategy, multiple behavior probability values ​​are judged step by step to determine the target driving behavior.

[0144] Optionally, the multiple behavior probability values ​​include an emergency braking probability value and a soft braking probability value, and the control unit 503 is specifically configured to:

[0145] Based on the emergency braking probability value and the first preset threshold, determining whether the target driving behavior is an emergency braking behavior;

[0146] If the target driving behavior is not an emergency braking behavior, whether the target driving behavior is a soft braking behavior is determined based on the soft braking probability value and the second preset threshold.

[0147] For the convenience of description, the above parts are divided into each unit module (or module) according to the function and described separately. Of course, when implementing the present application, the functions of each unit (or module) can be implemented in the same one or more software or hardware. It can be understood by those skilled in the art that various aspects of the present application can be implemented as a system, method or program product. Therefore, various aspects of the present application can be specifically implemented in the following forms, namely: a complete hardware implementation method, a complete software implementation method (including firmware, microcode, etc.), or an implementation method combining hardware and software, which can be collectively referred to as "circuit", "module" or "system" here.

[0148] The device can be used to execute the methods shown in the various embodiments of the present application. Therefore, for the functions that can be implemented by the various functional modules of the device, reference can be made to the description of the aforementioned embodiments and no further details will be given.

[0149] See also Figure 6 As shown, based on the same technical concept, the embodiment of the present application further provides a computer device 60. In one embodiment, the computer device may be a vehicle control device. Figure 6 As shown, it includes a memory 601 , a communication module 603 and one or more processors 602 .

[0150] The memory 601 is used to store computer programs executed by the processor 602. The memory 601 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0151] The memory 601 may be a volatile memory, such as a random-access memory (RAM); the memory 601 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 601 may be any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 601 may be a combination of the above memories.

[0152] The processor 602 may include one or more central processing units (CPU) or digital processing units, etc. The processor 602 is used to implement the above-mentioned vehicle control method based on mixed brain-muscle signals when calling the computer program stored in the memory 601 .

[0153] The communication module 603 is used to communicate with terminal devices such as sensors, vehicle controllers or other servers.

[0154] The specific connection medium between the memory 601, the communication module 603 and the processor 602 is not limited in the embodiment of the present application. Figure 6 In the embodiment, the memory 601 and the processor 602 are connected via a bus 604. The bus 604 is connected to the processor 602 via a bus 604. Figure 6 The connections between the other components are only for illustration and are not intended to be limiting. The bus 604 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Figure 6 The diagram shows that only one thick line is used, but this does not mean that there is only one bus or only one type of bus.

[0155] The memory 601 stores computer storage media, which stores computer executable instructions. The computer executable instructions are used to implement the vehicle control method based on mixed brain-muscle signals of the embodiments of the present application. The processor 602 is used to execute the vehicle control method based on mixed brain-muscle signals of the above-mentioned embodiments.

[0156] Based on the same inventive concept, an embodiment of the present application also provides a storage medium, which stores a computer program. When the computer program runs on a computer, the computer executes the steps of the vehicle control method based on hybrid brain-muscle signals according to various exemplary embodiments of the present application described above in this specification.

[0157] In some possible embodiments, various aspects of the vehicle control method based on hybrid brain-muscle signals provided in the present application can also be implemented in the form of a computer program product, which includes a computer program. When the program product is run on a computer device, the computer program is used to enable the computer device to execute the steps of the vehicle control method based on hybrid brain-muscle signals according to various exemplary embodiments of the present application described above in this specification. For example, the computer device can execute the steps of each embodiment.

[0158] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0159] The program product of the embodiment of the present application may adopt a portable compact disk read-only memory (CD-ROM) and include a computer program, and can be run on a computer device. However, the program product of the present application is not limited thereto. In the present application, the readable storage medium may be any tangible medium containing or storing a program, and the computer program included therein may be used by or in combination with a command execution system, apparatus or device.

[0160] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with a command execution system, apparatus, or device.

[0161] The computer program embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0162] Computer programs for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages.

[0163] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0164] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0165] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0166] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0167] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A vehicle control method based on mixed brain-muscle signals, characterized in that: The method comprises: Acquire a mixed brain-muscle signal set of a target object, and perform feature processing on the mixed brain-muscle signal set to obtain target EEG features and target myoelectric features; the mixed brain-muscle signal set includes multiple EEG signals and multiple myoelectric signals of the target object; Based on a preset hybrid strategy, driving behavior detection is performed on the target EEG feature and the target EMG feature to obtain a plurality of behavior probability values; each behavior probability value corresponds to a preset driving behavior; Based on the multiple behavior probability values, a target driving behavior of the target object is determined, and based on the target driving behavior, the vehicle is controlled accordingly.

2. The method according to claim 1, characterized in that When the preset hybrid strategy is a feature hybrid strategy, the driving behavior detection is performed on the target EEG feature and the target EMG feature based on the preset hybrid strategy to obtain multiple behavior probability values, including: Performing feature mixing processing on the target EEG feature and the target EMG feature to obtain mixed EEG feature; The brain-muscle features are input into a plurality of behavior detection models to obtain corresponding behavior probability values, respectively, and each behavior detection model corresponds to a preset driving behavior.

3. The method according to claim 1, characterized in that When the preset hybrid strategy is a classification hybrid strategy, the driving behavior detection is performed on the target EEG feature and the target EMG feature based on the preset hybrid strategy to obtain multiple behavior probability values, including: Inputting the target EEG feature and the target EMG feature into an EEG classifier and an EMG classifier, respectively, to obtain an EEG classification set and an EMG classification set; the EEG classification set and the EMG classification set both contain prediction probability values ​​corresponding to each preset driving behavior; Based on a preset hybrid sub-strategy, the results of the EEG classification set and the EMG classification set are judged to obtain a plurality of behavior probability values; each behavior probability value corresponds to a preset driving behavior.

4. The method according to claim 3, characterized in that The preset hybrid sub-strategy is used to judge the results of the EEG classification set and the EMG classification set to obtain multiple behavior probability values, including: For each preset driving behavior, the predicted probability values ​​in the EEG classification set and the EMG classification set are calculated to obtain the behavior probability value of each preset driving behavior; or, For each preset driving behavior, a behavior probability value of each preset driving behavior is obtained by performing or calculating the predicted probability values ​​in the EEG classification set and the EMG classification set.

5. The method according to claim 1, characterized in that When the preset hybrid strategy is a Bayesian hybrid strategy, the driving behavior detection is performed on the target EEG feature and the target EMG feature based on the preset hybrid strategy to obtain multiple behavior probability values, including: Based on the prior probability of each preset driving behavior, the conditional probability of the target EEG feature and the target EMG feature is calculated; the prior probability represents the probability of each preset driving behavior occurring when there is no EEG signal and EMG signal; the conditional probability represents the probability of each EEG signal and EMG signal occurring under the corresponding preset driving behavior; Based on the obtained conditional probabilities, the posterior probability of each preset driving behavior is calculated; the posterior probability represents the probability of each preset driving behavior occurring under any combination of EEG signals and EMG signals; A maximum a posteriori probability value is determined from the obtained multiple posterior probabilities, and the maximum a posteriori probability value is used as the behavior probability value of the corresponding preset driving behavior.

6. The method according to claim 1, characterized in that The determining the target driving behavior of the target object based on the multiple behavior probability values ​​includes: Determine a maximum behavior probability value from the multiple behavior probability values, and determine a preset driving behavior corresponding to the maximum behavior probability value as the target driving behavior; or, Based on the decision tree classification strategy, the multiple behavior probability values ​​are judged step by step to determine the target driving behavior.

7. The method according to claim 6, characterized in that The multiple behavior probability values ​​include an emergency braking probability value and a soft braking probability value. Then, the decision tree classification strategy is used to judge the multiple behavior probability values ​​step by step to determine the target driving behavior, including: Based on the emergency braking probability value and a first preset threshold, determining whether the target driving behavior is an emergency braking behavior; If the target driving behavior is not the emergency braking behavior, whether the target driving behavior is a soft braking behavior is determined based on the soft braking probability value and a second preset threshold.

8. A vehicle control device based on mixed brain-muscle signals, characterized in that: The device comprises: A processing unit, used to obtain a mixed brain-muscle signal set of a target object, and perform feature processing on the mixed brain-muscle signal set to obtain a target EEG feature and a target EMG feature; the mixed brain-muscle signal set includes a plurality of EEG signals and a plurality of EMG signals of the target object; A detection unit, configured to perform driving behavior detection on the target EEG feature and the target EMG feature based on a preset hybrid strategy to obtain a plurality of behavior probability values; each behavior probability value corresponds to a preset driving behavior; A control unit is used to determine a target driving behavior of the target object based on the multiple behavior probability values, and to control the vehicle accordingly based on the target driving behavior.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

11. A computer program product comprising computer program instructions, characterized in that When the computer program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images