Driving intention perception method and device, computer equipment and readable storage medium

Abstract

The invention relates to the technical field of driving, in particular to a driving intention sensing method and device, computer equipment and a readable storage medium. The method comprises the following steps: acquiring a detected electromyographic signal and a detected electroencephalogram signal acquired based on an electrode array; the electrode support structure of each electrode unit in the electrode array is constructed based on a finite element analysis method, and the arrangement structure of the electrode units in the electrode array is obtained through analysis according to key electroencephalogram characteristics; inputting the detected electromyographic signals and electroencephalogram signals into a multi-source fusion model to obtain fusion features output by the multi-source fusion model; the multi-source fusion model is established based on a Boltzmann machine; inputting the fusion features into an intention perception model to obtain the detected driving intention output by the intention perception model; the intention perception model is constructed based on a random forest algorithm.

Description

Technical Field

[0001] The present application relates to the field of driving technology, and in particular to a driving intention perception method, device, computer equipment and readable storage medium. Background Art

[0002] Driving intention is a kind of consciousness generated by the driver based on his own driving experience when facing complex and changeable road conditions. This consciousness can drive the driver to drive the car in the correct way and thus complete the driving task well. Driving intention perception is the process of identifying and understanding these consciousnesses through technical means.

[0003] Traditional technology relies on EEG signals to sense driving intentions. EEG is an electrophysiological monitoring method that records brain electrical activity non-invasively. Electrodes are placed along the scalp to record the brain's spontaneous electrical activity over a period of time.

[0004] However, when acquiring EEG signals, although wet electrodes can provide better signal quality, they need to be applied with conductive glue during use, which not only increases the complexity of use, but also causes discomfort when worn for a long time, making them unsuitable for long-term driving intention monitoring. Although dry electrodes solve the problem of using conductive glue, their signal quality is often poor and they are easily affected by poor skin contact, resulting in signal distortion, which in turn affects the accurate identification of driving intentions. Therefore, they are in urgent need of improvement. Summary of the invention

[0005] Based on this, it is necessary to provide a driving intention perception method, device, computer equipment and readable storage medium that can improve the accuracy of driving intention recognition in response to the above technical problems.

[0006] In a first aspect, the present application provides a driving intention perception method, the method comprising:

[0007] Acquire the measured electromyographic signal and the measured electroencephalographic signal collected based on the electrode array; the electrode bracket structure of each electrode unit in the electrode array is constructed based on the finite element analysis method, and the arrangement structure between the electrode units in the electrode array is obtained according to the key electroencephalographic feature analysis;

[0008] The measured electromyographic signals and electroencephalographic signals are input into the multi-source fusion model to obtain the fusion features output by the multi-source fusion model; the multi-source fusion model is constructed based on the Boltzmann mechanism;

[0009] The fused features are input into the intention perception model, and the intention perception model outputs the driving intention being tested; the intention perception model is constructed based on the random forest algorithm.

[0010] In one embodiment, the electrode support structure of any electrode unit is constructed based on the finite element analysis method in the following manner:

[0011] According to the structural parameters of biological neurons, a geometric model corresponding to the initial support structure of the electrode unit is constructed;

[0012] The geometric model is input into the finite element analysis software, and material property information and boundary conditions are added to the geometric model in the finite element analysis software to obtain a structural model corresponding to the geometric model;

[0013] In the finite element analysis model, external loads are applied to the structural model to obtain verification results of the structural strength of the structural model;

[0014] According to the strength verification result of the structural model, the structural model is updated to obtain the electrode bracket structure of the electrode unit.

[0015] In one embodiment, the arrangement structure between the electrode units in the electrode array is obtained according to the key EEG feature analysis, using the following method:

[0016] Determine the distribution area of ​​the measured features according to the key EEG features; the key EEG features include at least one of Delta wave, Theta wave, Alpha wave, Beta wave, Gamma wave, event-related potential and steady-state visual evoked potential;

[0017] With the goal of the electrode array covering the characteristic distribution area, the position of each electrode unit in the electrode array is determined.

[0018] In one embodiment, the measured electromyographic signal and electroencephalographic signal are input into a multi-source fusion model to obtain fusion features output by the multi-source fusion model, including:

[0019] Extracting the electromyographic features corresponding to the measured electromyographic signals and the electroencephalographic features corresponding to the electroencephalographic signals;

[0020] Input the electromyographic features and electroencephalographic features into the multi-source fusion model to obtain the fusion features output by the multi-source fusion model;

[0021] The electromyographic features at least include edge features, texture features, local pattern features, time-dependent features, dynamic change features and sequence pattern features in the electromyographic signal;

[0022] Local pattern features are used to identify different muscle activity states;

[0023] The time-dependent feature characterizes the correlation and change trend of EMG signals at different time points;

[0024] The sequence pattern feature characterizes the repetitive or periodic pattern in the time series of the EMG signal;

[0025] The EEG features at least include the time domain features, frequency domain features, spatial features and nonlinear features of the EEG signal.

[0026] In one embodiment, extracting the electromyographic feature corresponding to the measured electromyographic signal includes:

[0027] Through the two-dimensional convolutional neural network, edge features, texture features, and local pattern features in the electromyographic signal are extracted;

[0028] Through the long short-term memory network, the time-dependent features, dynamic change features and sequence pattern features in the electromyographic signals are extracted.

[0029] In one embodiment, extracting EEG features corresponding to the EEG signal includes:

[0030] Determine the time domain characteristics of the EEG signal according to the statistics of the EEG signal; the statistics include mean, variance and standard deviation;

[0031] By using spectrum analysis, the EEG signal is converted from the time domain to the frequency domain, and the frequency components and power spectrum of the EEG signal in the frequency domain are used as the frequency domain features of the EEG signal;

[0032] Obtain the correlation between the potentials of different brain regions and determine the spatial characteristics of EEG signals; the spatial characteristics of EEG signals are used to characterize the spatial correlation between different brain regions;

[0033] The nonlinear characteristics of EEG signals are extracted through nonlinear dynamics analysis.

[0034] In a second aspect, the present application also provides a driving intention perception device, comprising:

[0035] A signal acquisition module is used to acquire the measured electromyographic signal and the measured electroencephalographic signal collected based on the electrode array; the electrode bracket structure of each electrode unit in the electrode array is constructed based on the finite element analysis method, and the arrangement structure between the electrode units in the electrode array is obtained based on the key electroencephalographic feature analysis;

[0036] The signal fusion module is used to input the measured electromyographic signal and electroencephalographic signal into the multi-source fusion model to obtain the fusion features output by the multi-source fusion model; the multi-source fusion model is constructed based on the Boltzmann mechanism;

[0037] The intention perception determination module is used to input the fusion features into the intention perception model to obtain the driving intention being tested as output by the intention perception model; the intention perception model is constructed based on the random forest algorithm.

[0038] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0039] Acquire the measured electromyographic signal and the measured electroencephalographic signal collected based on the electrode array; the electrode bracket structure of each electrode unit in the electrode array is constructed based on the finite element analysis method, and the arrangement structure between the electrode units in the electrode array is obtained according to the key electroencephalographic feature analysis;

[0040] The measured electromyographic signals and electroencephalographic signals are input into the multi-source fusion model to obtain the fusion features output by the multi-source fusion model; the multi-source fusion model is constructed based on the Boltzmann mechanism;

[0041] The fused features are input into the intention perception model to obtain the driving intention being tested as the output of the intention perception model; the intention perception model is constructed based on the random forest algorithm.

[0042] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0043] Acquire the measured electromyographic signal and the measured electroencephalographic signal collected based on the electrode array; the electrode bracket structure of each electrode unit in the electrode array is constructed based on the finite element analysis method, and the arrangement structure between the electrode units in the electrode array is obtained according to the key electroencephalographic feature analysis;

[0044] The measured electromyographic signals and electroencephalographic signals are input into the multi-source fusion model to obtain the fusion features output by the multi-source fusion model; the multi-source fusion model is constructed based on the Boltzmann mechanism;

[0045] The fused features are input into the intention perception model to obtain the driving intention being tested as the output of the intention perception model; the intention perception model is constructed based on the random forest algorithm.

[0046] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0047] Acquire the measured electromyographic signal and the measured electroencephalographic signal collected based on the electrode array; the electrode bracket structure of each electrode unit in the electrode array is constructed based on the finite element analysis method, and the arrangement structure between the electrode units in the electrode array is obtained according to the key electroencephalographic feature analysis;

[0048] The measured electromyographic signals and electroencephalographic signals are input into the multi-source fusion model to obtain the fusion features output by the multi-source fusion model; the multi-source fusion model is constructed based on the Boltzmann mechanism;

[0049] The fused features are input into the intention perception model, and the intention perception model outputs the driving intention being tested; the intention perception model is constructed based on the random forest algorithm.

[0050] The above-mentioned driving intention perception method, device, computer equipment and readable storage medium collect the measured EEG signals based on a specially designed electrode array. The electrode bracket structure of each electrode unit of the electrode array is optimized by finite element analysis to ensure good contact and signal transmission. The arrangement structure of the electrode array is obtained based on the analysis of key EEG features to maximize the collection of EEG information related to driving intention. At the same time, the measured electromyographic signal (EMG) is obtained, which reflects the muscle activity and is directly related to the driving operation. At the same time, the acquired electromyographic signal and EEG signal are input into the multi-source fusion model. The model is based on the Boltzmann machine, which can learn the complex relationship between the two signals and extract fusion features. This fusion feature integrates the useful information in the electromyographic and EEG signals, and improves the accuracy of driving intention recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 A schematic diagram of a process of a driving intention perception method in one embodiment;

[0053] Figure 2 It is a schematic flow chart of steps for constructing an electrode support structure of any electrode unit based on a finite element analysis method in one embodiment;

[0054] Figure 3 It is a flowchart of the steps of obtaining the arrangement structure between the electrode units in the electrode array according to the key EEG feature analysis in one embodiment;

[0055] Figure 4 A schematic flow chart of a step of obtaining fusion features output by a multi-source fusion model in one embodiment;

[0056] Figure 5 A schematic diagram of a flow chart of a step of extracting EEG features corresponding to an EEG signal in one embodiment;

[0057] Figure 6 is a structural block diagram of a driving intention perception device in one embodiment;

[0058] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] In an exemplary embodiment, Figure 1 As shown, a driving intention perception method is provided, and the method is applied to a computer device as an example for explanation, including:

[0061] S101, obtaining the measured electromyographic signal and the measured electroencephalographic signal collected based on the electrode array.

[0062] It is understandable that the electrodes are placed on predetermined muscle positions, which may be key muscle groups related to driving intention, such as arms, legs, etc. The electromyographic acquisition device is started to record the electrical signals generated by muscle activity, ensuring the stability and continuity of the signals during the acquisition process, and avoiding signal interruption due to electrode shedding or movement.

[0063] Among them, the electrode support structure of each electrode unit in the electrode array is constructed based on the finite element analysis method, and the arrangement structure between the electrode units in the electrode array is obtained according to the key EEG characteristic analysis.

[0064] It is understandable that in driving intention perception technology, the design of the electrode support structure is crucial to the quality of EEG signal collection. In order to ensure that the electrode can stably and accurately collect the electrical signals generated by brain activity, this embodiment uses the finite element analysis method (FEA) to construct the electrode support structure.

[0065] Among them, the finite element analysis method is a numerical analysis method, which divides a complex structure or system into a finite number of interconnected units or nodes, and then simulates the behavior of the entire structure or system by analyzing and calculating these units or nodes. In this embodiment, the finite element analysis method is used to perform mechanical simulation of the electrode holder structure to evaluate its stability and flexibility under different stress conditions. Through finite element analysis, key parameters such as deformation and stress distribution of the electrode holder under stress can be accurately calculated, thereby guiding the optimal design of the electrode holder structure. The optimized electrode holder structure can ensure close contact between the electrode and the scalp, reduce signal distortion and noise interference, and improve the acquisition quality of EEG signals.

[0066] In addition, the arrangement structure of the electrode array also has an important influence on the collection of EEG signals. In order to accurately capture the EEG signals related to driving intentions, this embodiment adopts a key EEG feature analysis method to guide the arrangement design of the electrode array. Key EEG feature analysis is a method of identifying EEG features related to specific tasks or intentions by preprocessing, feature extraction and classification of EEG signals. Based on the results of the key EEG feature analysis, a more reasonable electrode array arrangement structure can be designed. By optimizing the layout position and number of electrodes, it can be ensured that the electrode array can accurately capture EEG signals related to driving intentions, and improve the accuracy and reliability of intention perception. At the same time, the optimized arrangement of the electrode array can also reduce unnecessary information redundancy and noise interference, and reduce computational complexity and resource consumption.

[0067] S102, inputting the measured electromyographic signal and electroencephalographic signal into the multi-source fusion model to obtain the fusion features output by the multi-source fusion model.

[0068] Among them, the multi-source fusion model is constructed based on the Boltzmann mechanism.

[0069] It is understandable that the Boltzmann Machine is an energy-based random neural network model suitable for dealing with complex data fusion problems. The Boltzmann Machine plays a key role in inputting the measured electromyographic signals (EMG) and electroencephalographic signals (EEG) into the multi-source fusion model and obtaining the fusion features.

[0070] Specifically, according to actual needs, define the structure of the Boltzmann machine, including the number of nodes in the input layer, hidden layer, and output layer, and the connection relationship between the nodes. The number of nodes in the input layer should match the total number of channels of the measured EMG and EEG signals. Randomly initialize the weight matrix and bias vector of the Boltzmann machine. Set training parameters such as learning rate and number of iterations.

[0071] The measured EMG and EEG signals are preprocessed, including filtering, denoising, feature extraction and other steps to ensure the quality and consistency of the signals. The preprocessed signals are converted into a format suitable for Boltzmann machine processing, such as normalizing the signal values ​​to a specific range. The preprocessed EMG and EEG signals are used as input data and mapped to the input layer nodes of the Boltzmann machine respectively.

[0072] According to the definition of the Boltzmann machine, the energy function value in the current state is calculated. The energy function can be related to the state and weight of the node and is used to measure the instability of the system. According to Gibbs Sampling or other sampling methods, the state of the node in the Boltzmann machine is iteratively updated. In each iteration, according to the state and weight of the current node, the conditional probability distribution of other nodes is calculated, and new state values ​​are sampled from them.

[0073] According to the update of node status, the weight matrix and bias vector of the Boltzmann machine are adjusted to reduce the energy function value and stabilize the system. The weights can be adjusted by using optimization methods such as the Contrastive Divergence (CD) algorithm. After multiple iterations, the hidden layer nodes of the Boltzmann machine will contain the feature information of the fused EMG and EEG signals. The fused features can be extracted from the hidden layer nodes for subsequent intent understanding or classification tasks.

[0074] S103, inputting the fused features into the intention perception model to obtain the driving intention being measured as output by the intention perception model.

[0075] Among them, the intention perception model is constructed based on the random forest algorithm.

[0076] It is understandable that random forest is an integrated learning method that improves the accuracy and stability of classification or regression tasks by constructing multiple decision trees and integrating their output results. During the training process of each decision tree, random sampling (such as Bootstrap sampling) and random selection of features are used to increase the diversity of the model. Multi-layer random forests are composed of multiple levels of random forests, each layer receives the output of the previous layer as input and outputs its classification or regression results. In this embodiment, the first layer of random forest is used to extract preliminary classification features from the fused features, and subsequent layers further refine the classification results.

[0077] Specifically, the preprocessed EMG and EEG signals are fused through the Boltzmann machine to extract fusion features. Ensure that the data format and range of the fusion features match the input requirements of the multi-layer random forest model.

[0078] The fused features are then passed as input data to the first layer of a multi-layer random forest model. In each layer, the random forest makes decisions and classifications based on the input features and passes the results to the next layer. The output of the final layer of random forest is the driving intention being measured. Based on the output of the model, it can be determined whether the driver intends to accelerate, brake, turn, or perform other driving operations.

[0079] Exemplarily, feature enhancement is performed through a multi-layer perceptron (MLP) training unit and an SG parameter calculator. After collecting training and test data, a multi-layer perceptron (MLP) is used for training, and the sample data parameters are optimized through an SG parameter calculator, such as the formula

[0080]

[0081] The weight Determined by the polynomial fit and fixed for each sliding window. These weights ensure that the polynomial fit within a given window is optimal.

[0082] In addition, a feature classification and feature storage system is established. The feature classification module uses the neural network layer to process the training data and evaluates the classification accuracy by calculating the KL divergence. The feature storage module uses Kafka to receive the data stream, performs real-time analysis through the stream engine, and stores the processed data in the data warehouse. The system also includes a SQL engine, a batch processing engine, and a machine learning engine to achieve classified storage and unified management of data.

[0083] In the above-mentioned driving intention perception method, the measured EEG signal is collected based on a specially designed electrode array. The electrode bracket structure of each electrode unit of the electrode array is optimized by the finite element analysis method to ensure good contact and signal transmission. The arrangement structure of the electrode array is obtained based on the analysis of key EEG features to maximize the collection of EEG information related to driving intention. At the same time, the measured electromyographic signal (EMG) is obtained, which reflects the muscle activity and is directly related to the driving operation. At the same time, the acquired electromyographic signal and EEG signal are input into the multi-source fusion model. The model is built based on the Boltzmann machine, which can learn the complex relationship between the two signals and extract fusion features. This fusion feature combines the useful information in the electromyographic and EEG signals, and improves the accuracy of driving intention recognition.

[0084] In an exemplary embodiment, Figure 2 As shown, the electrode support structure of any electrode unit is constructed based on the finite element analysis method in the following manner:

[0085] S201, constructing a geometric model corresponding to the initial support structure of the electrode unit according to the structural parameters of the biological neuron.

[0086] Among them, the morphology and structural characteristics of neurons in organisms (such as humans) are studied, especially the branching pattern, diameter variation, and overall layout of neurons. Relevant data on neuronal structure are collected, including branch length, diameter, branch angle, etc. Structural parameters that are critical to electrode holder design are extracted from the collected data.

[0087] Specifically, select appropriate computer-aided design (CAD) or finite element analysis (FEA) software as the modeling platform. In the modeling software, according to the extracted biological neuron structural parameters, start to build the geometric model corresponding to the initial support structure of the electrode unit. Use the modeling tools of the software, such as lines, curves, surfaces, etc., to simulate the branching pattern of neurons. In the process of building the model, continuously optimize the details of the model, such as the smoothness of the branches, the gradual change of the diameter, etc.

[0088] Mesh the constructed geometric model and decompose it into smaller units (such as triangular or quadrilateral meshes). Define the corresponding material properties in the model according to the material of the electrode holder (such as metal, plastic, etc.). Including parameters such as the elastic modulus, Poisson's ratio, density, etc. of the material. Apply appropriate boundary conditions and loads in the model according to the actual use of the electrode holder. For example, the contact pressure between the electrode holder and the skin, the gravity of the holder itself, etc. can be simulated.

[0089] Run finite element analysis in the software to calculate the model's stress, strain and other mechanical responses. Observe the analysis results and check whether the model has problems such as stress concentration and excessive deformation. According to the analysis results, optimize the geometric model of the electrode holder. You can adjust the branch diameter, length, angle and other parameters to improve the mechanical properties of the model. Repeat the finite element analysis until you get a satisfactory design result.

[0090] S202, inputting the geometric model into finite element analysis software, and adding material property information and boundary conditions to the geometric model in the finite element analysis software to obtain a structural model corresponding to the geometric model.

[0091] Specifically, open the finite element analysis software and use the software's import function to import the electrode holder geometry model (such as a CAD file) previously constructed based on the biological neuron structure parameters into the software. Ensure that the imported geometry model maintains the original size, shape, and accuracy.

[0092] In the finite element analysis software, select a suitable material library, which contains the mechanical properties of various common materials. According to the actual material of the electrode stent (such as titanium alloy, stainless steel, biocompatible plastic, etc.), select or customize the corresponding material properties in the material library. The key properties that need to be defined include elastic modulus (a parameter describing the stiffness of the material), Poisson's ratio (a parameter describing the lateral deformation of the material when subjected to force), density (the ratio of the weight to the volume of the material), etc. Apply the defined material properties to the imported geometric model.

[0093] Determine the type of boundary conditions that need to be applied based on the actual use and stress state of the electrode holder. Common boundary conditions include fixed constraints, displacement constraints, force or pressure loads, etc. In the finite element analysis software, this step is completed by selecting specific parts or nodes on the model and setting the corresponding boundary conditions. For example, if one end of the electrode holder is fixed to the skin, a fixed constraint can be applied to that end; if the holder is squeezed by the muscle, a pressure load can be applied to the corresponding position.

[0094] In the finite element analysis software, mesh the imported geometric model. Meshing is to decompose the model into smaller units (such as triangles, quadrilaterals or tetrahedrons) for more accurate mechanical analysis. According to the complexity and accuracy requirements of the model, select the appropriate meshing method and density. After confirming that the geometric model, material property information and boundary conditions are correct, the software will automatically generate a structural model. This model will be used for subsequent mechanical simulation and analysis.

[0095] S203, applying an external load to the structural model in the finite element analysis model to obtain a verification result of the structural strength of the structural model.

[0096] Optionally, ensure that the geometric model of the electrode holder has been constructed according to the structural parameters of the biological neuron, and successfully imported into the finite element analysis software, and the correct material property information and boundary conditions have been added to generate the structural model. According to the actual stress conditions of the electrode holder in use, select the appropriate load type. Common load types include pressure, tension, bending force, etc.

[0097] Define the size, direction and point of action of the load according to the design requirements and actual use of the electrode holder. Ensure that the load parameters are consistent with the actual situation and can truly reflect the stress state of the electrode holder in use. In the finite element analysis software, apply external loads by selecting specific parts or nodes on the model and setting the corresponding load parameters. Ensure that the load is correctly applied to the model and is consistent with the actual situation.

[0098] According to the analysis requirements, set the simulation parameters, such as solver type, number of iterations, convergence criteria, etc. Ensure that the simulation parameters are reasonable and accurate analysis results can be obtained. In the finite element analysis software, start the simulation analysis process. The software will perform mechanical simulation calculations based on information such as the structural model, material properties, boundary conditions, and external loads. After the simulation is completed, export the analysis results from the finite element analysis software. These results can include stress distribution diagrams, strain distribution diagrams, displacement distribution diagrams, etc., which are used to evaluate the structural strength.

[0099] S204, updating the structural model according to the strength verification result of the structural model to obtain the electrode support structure of the electrode unit.

[0100] Specifically, the structural strength of the electrode bracket is evaluated based on the analysis results. By comparing parameters such as stress, strain and displacement with the allowable values ​​or design standards of the material, it is determined whether the electrode bracket meets the strength requirements. If the analysis results show that some parts of the electrode bracket have excessive stress or strain, or the displacement exceeds the allowable range, an optimized design is required to improve the structural strength.

[0101] In an exemplary embodiment, Figure 3 As shown, the arrangement structure between the electrode units in the electrode array is obtained based on the key EEG feature analysis, using the following method:

[0102] S301, determining a measured characteristic distribution area according to key EEG characteristics.

[0103] Among them, the key EEG characteristics include at least one of Delta wave, Theta wave, Alpha wave, Beta wave, Gamma wave, event-related potential and steady-state visual evoked potential.

[0104] It is understandable that Delta waves are mainly distributed in the deep areas of the cerebral cortex and are related to deep sleep and slow-wave sleep. In the design of electrode arrays, it is possible to consider arranging electrode units at positions corresponding to the deep layers of the cerebral cortex to capture Delta wave signals.

[0105] Theta waves: They mainly appear in the frontal and parietal regions and are related to cognitive processes such as attention, learning, and memory. Therefore, in the electrode array, the electrode units in these areas should be arranged more densely to improve the accuracy of theta wave signal acquisition.

[0106] Alpha wave: It is most significant in the occipital region and is related to relaxation. In the design of electrode arrays, the spacing between electrode units and the visual cortical activity under closed eyes need to be considered. In the design of electrode arrays, electrode units can be arranged preferentially in the occipital region to capture Alpha wave signals.

[0107] Beta waves: Mainly distributed in the frontal lobe and precentral gyrus, they are associated with concentration, alertness, and higher cognitive functions. In the electrode array, the electrode units in these areas should also be arranged relatively densely.

[0108] Gamma waves: They are distributed in many areas of the cerebral cortex and are related to sensory processing, cognitive processes, memory consolidation, etc. Since gamma waves have a higher frequency and a faster propagation speed, the bureau ensures that gamma wave signals can be captured.

[0109] Event-related potential (ERP): ERP is an EEG signal related to a specific event, and its distribution area depends on the type of event and the area of ​​the brain that processes the event. In the design of the electrode array, the layout of the electrode unit needs to be determined according to the characteristics of the experimental task and the ERP component.

[0110] Steady-state visual evoked potential (SSVEP): SSVEP is an EEG signal associated with visual stimulation, mainly distributed in the occipital and parietal regions. In the electrode array design, the electrode units in these areas should be arranged densely and accurately enough to capture the SSVEP signal.

[0111] It is understandable that the above characteristics have different distribution patterns and physiological significance in the cerebral cortex, which is of great significance for understanding brain function and state.

[0112] Exemplarily, key features may be determined by a feature selection method, the specific process being:

[0113] Firstly, the physical and physiological characteristics information related to the optimization of EEG electrode array under different driving intentions is collected, and the collected data is preprocessed.

[0114] Then, the recursive feature elimination (RFE) method is used to gradually remove unimportant features and retain key features. The steps are as follows:

[0115] ①The initial feature set is defined as ;

[0116] ② Define the number of features to be removed in each iteration .

[0117] ③For each iteration Use the current feature set first Train the model, calculate the importance of each feature, sort the features by their importance, remove the least important ones features, and obtain a new feature set : .

[0118] Repeat the above steps to obtain the required key features.

[0119] Finally, the genetic algorithm (GA) is used to select the optimal feature combination, and the optimal feature subset is determined as the key EEG feature by evaluating and screening the features.

[0120] S302, with the electrode array covering the characteristic distribution area as the goal, determining the position of each electrode unit in the electrode array.

[0121] Optionally, a distribution area corresponding to the key feature is determined, and the distribution area can be obtained through statistical analysis and imaging technology (such as electroencephalogram, functional magnetic resonance imaging, etc.).

[0122] When designing an electrode array, certain principles need to be followed. For example, the number and density of electrode units should be determined based on the area and shape of the characteristic distribution region; the spacing between electrode units should be small enough to ensure that subtle changes in EEG signals can be captured; at the same time, the layout of electrode units should also take into account the curvature and structure of the cerebral cortex to avoid signal distortion caused by poor contact between the electrode and the scalp.

[0123] With the goal of the electrode array covering the characteristic distribution area, the position of each electrode unit on the electrode array is preliminarily determined according to the shape and size of the characteristic distribution area. This can be achieved by drawing the outline of the characteristic distribution area on the electrode array and arranging the electrode units evenly or as needed within the outline.

[0124] When determining the position of the electrode unit, the orientation and angle of the electrode unit also need to be considered. Due to the curvature and structure of the cerebral cortex, the orientation and angle of the electrode unit may affect the signal acquisition effect. Therefore, the orientation and angle of the electrode unit should be adjusted according to the actual shape of the cerebral cortex and the characteristics of the characteristic distribution area to ensure that it can best capture the EEG signal.

[0125] In an exemplary embodiment, Figure 4 As shown, the measured electromyographic signals and electroencephalographic signals are input into the multi-source fusion model to obtain the fusion features output by the multi-source fusion model, including:

[0126] S401, extracting the electromyographic features corresponding to the electromyographic signals and the electroencephalographic features corresponding to the electroencephalographic signals.

[0127] Optionally, a signal processing circuit is used to filter the electromyographic signal and the electroencephalographic signal.

[0128] Specifically, the signal processing circuit uses a signal processing circuit with ramp wave technology to perform multi-stage filtering and noise reduction on the signal. The signal processing circuit includes a differential amplifier, a low-pass power notch filter, a buffer circuit, a gain amplifier circuit, a low-pass filter circuit and an analog-to-digital conversion circuit. Each module is provided with a stable voltage by a reference voltage generator and works in coordination through a non-overlapping clock.

[0129] S402, inputting the electromyographic features and the electroencephalographic features into the multi-source fusion model to obtain the fusion features output by the multi-source fusion model.

[0130] Among them, the electromyographic features at least include edge features, texture features, local pattern features, time-dependent features, dynamic change features and sequence pattern features in the electromyographic signal.

[0131] It can be understood that edge features refer to the features of mutation points or important turning points in EMG signals. These points can represent the beginning, end or significant changes in muscle activity. Texture features describe the subtle change patterns of EMG signals in time and amplitude, similar to the texture in an image. Local pattern features are used to identify different muscle activity states, such as rest, contraction and relaxation. Time-dependent features characterize the correlation and change trend of EMG signals at different time points. Dynamic change features describe the rapid changes of EMG signals over time, such as speed, acceleration, etc. Sequence pattern features characterize repetitive or periodic patterns in the time series of EMG signals.

[0132] In one possible implementation, extracting the electromyographic features corresponding to the electromyographic signal under test includes: extracting edge features, texture features, and local pattern features in the electromyographic signal through a two-dimensional convolutional neural network. Extracting time-dependent features, dynamic change features, and sequence pattern features in the electromyographic signal through a long short-term memory network.

[0133] Exemplarily, the EEG signals are filtered, denoised and normalized by using a bidirectional long short-term memory network (BiLSTM) to construct an EEG lead network, and the time-frequency characteristics are analyzed using Stockwell transform and wavelet transform to generate a time-frequency characteristic matrix; the EEG signals are preprocessed by equidistant high-azimuth projection and partially overlapping windows to generate a space-time characteristic matrix.

[0134] In one possible implementation, the EEG features include at least time domain features, frequency domain features, spatial features, and nonlinear features of the EEG signal.

[0135] It can be understood that time domain features refer to the characteristics of EEG signals that can be directly observed in time, such as amplitude, mean, variance, peak value, and waveform shape. Frequency domain features refer to the distribution characteristics of EEG signals in frequency, such as the power spectrum of different frequency components, frequency bandwidth, etc. Nonlinear features refer to the nonlinear dynamic characteristics existing in EEG signals, such as chaos, fractal, self-similarity, etc.

[0136] like Figure 5 As shown, the EEG features corresponding to the EEG signal are extracted, including:

[0137] S501, determining the time domain characteristics of the EEG signal according to the statistics of the EEG signal.

[0138] Among them, statistics include mean, variance and standard deviation.

[0139] S502, converting the EEG signal from the time domain to the frequency domain by spectrum analysis, and taking the frequency components and power spectrum of the EEG signal in the frequency domain as the frequency domain features of the EEG signal.

[0140] S503, obtaining the correlation relationship between the potentials of different brain regions and determining the spatial characteristics of the EEG signal.

[0141] Among them, the spatial characteristics of EEG signals are used to characterize the spatial correlation between different brain regions.

[0142] S504, extracting the nonlinear characteristics of the EEG signal through a nonlinear dynamics analysis method.

[0143] Exemplarily, the EEG signals are filtered, denoised and normalized by using a bidirectional long short-term memory network (BiLSTM) to construct an EEG lead network, and the time-frequency characteristics are analyzed using Stockwell transform and wavelet transform to generate a time-frequency characteristic matrix; the EEG signals are preprocessed by equidistant high-azimuth projection and partially overlapping windows to generate a space-time characteristic matrix.

[0144] Then, a 3D convolutional neural network (3DCNN) is used to process the time-frequency characteristic matrix and the time-space characteristic matrix to extract the time-frequency and time-space information of the EEG. A 3D convolutional neural network consists of multiple 3D convolutional layers, activation functions, 3D pooling layers, and fully connected layers. The 3D convolutional layers and activation functions are shown in the formula

[0145]

[0146] in It is The activation output of the layer, It is The weight of the layer, It is The bias of the layer, is the activation function; the 3D pooling layer is as follows

[0147]

[0148] The formula for the fully connected layer is as follows:

[0149]

[0150] Finally, a graph convolutional neural network (GCNN) is used to extract the structural information of EEG leads, and the extracted features are input into BiLSTM for processing and optimized with the attention mechanism to obtain high-precision EEG features.

[0151] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0152] Based on the same inventive concept, the embodiment of the present application also provides a driving intention perception device for implementing the above-mentioned driving intention perception method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more driving intention perception device embodiments provided below can refer to the limitations of the driving intention perception method above, and will not be repeated here.

[0153] In an exemplary embodiment, Figure 6 As shown, a driving intention perception device is provided, comprising:

[0154] The signal acquisition module 11 is used to acquire the measured electromyographic signal and the measured electroencephalographic signal collected based on the electrode array; the electrode support structure of each electrode unit in the electrode array is constructed based on the finite element analysis method, and the arrangement structure between the electrode units in the electrode array is obtained according to the key electroencephalographic feature analysis;

[0155] The signal fusion module 12 is used to input the measured electromyographic signal and electroencephalographic signal into the multi-source fusion model to obtain the fusion features output by the multi-source fusion model; the multi-source fusion model is constructed based on the Boltzmann mechanism;

[0156] The intention perception determination module 13 is used to input the fusion features into the intention perception model to obtain the driving intention to be tested output by the intention perception model; the intention perception model is constructed based on the random forest algorithm.

[0157] Each module in the above-mentioned driving intention perception device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0158] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a driving intention perception method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0159] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0160] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0161] Acquire the measured electromyographic signal and the measured electroencephalographic signal collected based on the electrode array; the electrode bracket structure of each electrode unit in the electrode array is constructed based on the finite element analysis method, and the arrangement structure between the electrode units in the electrode array is obtained according to the key electroencephalographic feature analysis;

[0162] The measured electromyographic signals and electroencephalographic signals are input into the multi-source fusion model to obtain the fusion features output by the multi-source fusion model; the multi-source fusion model is constructed based on the Boltzmann mechanism;

[0163] The fused features are input into the intention perception model to obtain the driving intention being tested as the output of the intention perception model; the intention perception model is constructed based on the random forest algorithm.

[0164] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0165] Acquire the measured electromyographic signal and the measured electroencephalographic signal collected based on the electrode array; the electrode bracket structure of each electrode unit in the electrode array is constructed based on the finite element analysis method, and the arrangement structure between the electrode units in the electrode array is obtained according to the key electroencephalographic feature analysis;

[0166] The measured electromyographic signals and electroencephalographic signals are input into the multi-source fusion model to obtain the fusion features output by the multi-source fusion model; the multi-source fusion model is constructed based on the Boltzmann mechanism;

[0167] The fused features are input into the intention perception model to obtain the driving intention being tested as the output of the intention perception model; the intention perception model is constructed based on the random forest algorithm.

[0168] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0169] Acquire the measured electromyographic signal and the measured electroencephalographic signal collected based on the electrode array; the electrode bracket structure of each electrode unit in the electrode array is constructed based on the finite element analysis method, and the arrangement structure between the electrode units in the electrode array is obtained according to the key electroencephalographic feature analysis;

[0170] The measured electromyographic signals and electroencephalographic signals are input into the multi-source fusion model to obtain the fusion features output by the multi-source fusion model; the multi-source fusion model is constructed based on the Boltzmann mechanism;

[0171] The fused features are input into the intention perception model to obtain the driving intention being tested as the output of the intention perception model; the intention perception model is constructed based on the random forest algorithm.

[0172] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0173] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0174] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A driving intention perception method, characterized in that: The method comprises: Acquire the measured electromyographic signal and the measured electroencephalographic signal collected based on the electrode array; the electrode support structure of each electrode unit in the electrode array is constructed based on the finite element analysis method, and the arrangement structure between the electrode units in the electrode array is obtained according to the key electroencephalographic feature analysis; Inputting the measured electromyographic signal and electroencephalographic signal into a multi-source fusion model to obtain fusion features output by the multi-source fusion model; the multi-source fusion model is constructed based on the Boltzmann mechanism; The fusion feature is input into the intention perception model, and the intention perception model outputs the tested driving intention; the intention perception model is constructed based on the random forest algorithm.

2. The method according to claim 1, characterized in that The electrode support structure of any electrode unit is constructed based on the finite element analysis method in the following way: According to the structural parameters of the biological neuron, a geometric model corresponding to the initial support structure of the electrode unit is constructed; Inputting the geometric model into finite element analysis software, and adding material property information and boundary conditions to the geometric model in the finite element analysis software to obtain a structural model corresponding to the geometric model; In the finite element analysis model, an external load is applied to the structural model to obtain a verification result of the structural strength of the structural model; According to the strength verification result of the structural model, the structural model is updated based on the strength verification result to obtain the electrode support structure of the electrode unit.

3. The method according to claim 2, characterized in that The arrangement structure between the electrode units in the electrode array is obtained according to the key EEG feature analysis, using the following method: Determining the distribution area of ​​the measured feature according to the key EEG feature; the key EEG feature includes at least one of Delta wave, Theta wave, Alpha wave, Beta wave, Gamma wave, event-related potential and steady-state visual evoked potential; With the goal of the electrode array covering the characteristic distribution area, the position of each electrode unit in the electrode array is determined.

4. The method according to claim 1, characterized in that The step of inputting the measured electromyographic signal and electroencephalographic signal into a multi-source fusion model to obtain fusion features output by the multi-source fusion model includes: Extracting the electromyographic features corresponding to the measured electromyographic signal and the electroencephalographic features corresponding to the electroencephalographic signal; Inputting the electromyographic features and the electroencephalographic features into a multi-source fusion model to obtain fusion features output by the multi-source fusion model; Wherein, the electromyographic features at least include edge features, texture features, local pattern features, time-dependent features, dynamic change features and sequence pattern features in the electromyographic signal; The local pattern features are used to identify different muscle activity states; The time-dependent feature characterizes the correlation and change trend of the electromyographic signal at different time points; The sequence pattern feature characterizes a repetitive pattern or a periodic pattern in the electromyographic signal time series; The EEG features at least include time domain features, frequency domain features, spatial features and nonlinear features of the EEG signal.

5. The method according to claim 4, characterized in that Extracting the electromyographic features corresponding to the electromyographic signal under test, including: Extracting edge features, texture features, and local pattern features from the electromyographic signal through a two-dimensional convolutional neural network; The time-dependent features, dynamic change features and sequence pattern features in the electromyographic signal are extracted through a long short-term memory network.

6. The method according to claim 4, characterized in that Extracting EEG features corresponding to the EEG signal includes: Determining the time domain characteristics of the EEG signal according to the statistics of the EEG signal; the statistics include mean, variance and standard deviation; The EEG signal is converted from the time domain to the frequency domain by spectrum analysis, and the frequency components and power spectrum of the EEG signal in the frequency domain are used as the frequency domain features of the EEG signal; Obtaining the correlation relationship between the potentials of different brain regions and determining the spatial characteristics of the EEG signal; the spatial characteristics of the EEG signal are used to characterize the spatial correlation between different brain regions; The nonlinear characteristics of the electroencephalogram signal are extracted by nonlinear dynamics analysis.

7. A driving intention sensing device, characterized in that: The device comprises: A signal acquisition module, used to acquire the measured electromyographic signal and the measured electroencephalographic signal collected based on the electrode array; the electrode support structure of each electrode unit in the electrode array is constructed based on the finite element analysis method, and the arrangement structure between the electrode units in the electrode array is obtained according to the key electroencephalographic feature analysis; A signal fusion module, used for inputting the measured electromyographic signal and electroencephalographic signal into a multi-source fusion model to obtain fusion features output by the multi-source fusion model; the multi-source fusion model is constructed based on the Boltzmann mechanism; The intention perception determination module is used to input the fusion feature into the intention perception model, so that the intention perception model outputs the driving intention under test; the intention perception model is constructed based on the random forest algorithm.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images