A driving state detection method based on dynamic trigger neurons

Abstract

The application relates to the technical field of driving state monitoring, and specifically provides a driving state detection method based on dynamic trigger neurons, which comprises the following steps: collecting time sequence tactile signals of a driver, dividing the time sequence tactile signals into multiple time sequence segments, respectively performing feature statistics and dimension reduction, constructing a total sequence of tactile feature vectors, and estimating the fatigue state of the driver from the total sequence of tactile feature vectors by using a fatigue state recognition model, wherein the model comprises an encoder, an attention generation module and a decoder; the encoder converts the continuous total sequence of tactile feature vectors into a pulse sequence, obtains enhanced features through the attention generation module, and finally decodes the enhanced features through the decoder to output a fatigue state classification result. The application introduces tactile information as a bridge for coupling of physiology-behavior-state, realizes accurate mapping of the tactile information and the state of the driver, and performs feature pulsing by using dynamic trigger neurons, so that the algorithm calculation amount is reduced.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of driving state monitoring, and particularly relates to a driving state detection method based on dynamic trigger neurons. BACKGROUND

[0002] With the continuous growth of the number of cars and the increasing complexity of road traffic, traffic safety has become the focus of social attention. Studies have shown that more than 90% of road accidents are caused by human errors, among which fatigue driving is one of the important reasons for accidents. Under the state of fatigue, the reaction speed, attention concentration and judgment ability of the driver will decrease significantly, greatly increasing the risk of driving. Therefore, the research on the monitoring of the fatigue state of the driver is of great significance for preventing traffic accidents.

[0003] The current methods for monitoring the fatigue of the driver mainly include three types:

[0004] The method for monitoring the fatigue state of the driver based on vehicle motion information identifies the fatigue state of the driver through vehicle operation data, but the vehicle operation data is difficult to directly reflect the physiological state of the driver, and is easily disturbed by the road environment to produce misjudgment.

[0005] The monitoring method based on physiological signals collects physiological characteristics such as electrocardiogram and brain waves through wearable devices, and identifies fatigue through changes in biological characteristics. The wearable sensor has driving interference to the driver and poor comfort, and is mainly suitable for the experimental stage and not suitable for actual popularization and application.

[0006] The method for monitoring the fatigue of the driver based on driving image identifies the face or action image of the driver through the in-vehicle camera monitoring device. This method is difficult to capture small actions, and the recognition effect is also easily affected by the driver's dress, lighting conditions and road bumping conditions.

[0007] The above monitoring methods need to establish the relationship between driving data, physiological signals or driving images and driving state through algorithms. The commonly used algorithms of the method for monitoring the fatigue state of the driver based on vehicle motion information include traditional machine learning models such as support vector machine (SVM) and random forest (Random Forest). These algorithms extract statistical features such as steering wheel corner variance, lane deviation frequency and lateral acceleration standard deviation to realize the pattern classification of driving behavior. However, this kind of method often depends on manual feature engineering, and has limited modeling ability for time series dynamic characteristics and long-range dependence relationship, and is easily disturbed by driving scenes, resulting in insufficient generalization performance.

[0008] The commonly used algorithm of the physiological signal based monitoring method includes a long short-term memory network (LSTM), a gated recurrent unit (GRU) and the like recurrent neural network, the calculation complexity of these short algorithms is high, the hardware resource requirement is large, and the physiological signal itself is susceptible to noise interference, which limits the deployment efficiency in the real vehicle environment.

[0009] The commonly used algorithm of the driver image based monitoring method is a convolutional neural network (CNN) such as ResNet, EfficientNet and the like, visual features such as eye movement and mouth state are extracted from the driver face image, but this method is not sensitive to early and subtle changes of the fatigue state.

[0010] The monitoring algorithm of the prior art generally has a large calculation amount, a large amount of invalid data calculation, the vehicle-mounted chip calculation unit has low precision and poor calculation ability, and it is difficult to meet the requirement of the vehicle-mounted chip calculation ability of the existing algorithm. Therefore, it is urgent to provide a driving state detection scheme which can overcome the drawbacks of the traditional scheme and has a low requirement on the calculation ability of the monitoring algorithm. SUMMARY

[0011] Therefore, the present application aims to provide a driving state detection method based on dynamic trigger neurons, which uses tactile information as the data source for driving state detection, constructs a fatigue state recognition model, and not only performs data dimension reduction through the Pearson analysis method in the data processing process, but also converts the feature input into a signal in the form of a pulse for information transmission by using the dynamic trigger neurons, thereby greatly reducing the calculation amount and power consumption, and solving the problem that the traditional fatigue state detection algorithm cannot run on the vehicle-mounted chip with weak calculation ability.

[0012] To achieve the above object, the technical scheme of the present application is as follows:

[0013] The present application provides a driving state detection method based on dynamic trigger neurons, comprising:

[0014] obtaining time sequence tactile signals collected by a pressure sensor array on a steering wheel and a seat of a vehicle;

[0015] dividing the time sequence tactile signals into a plurality of time sequence segments, performing feature statistics and dimension reduction on each time sequence segment, obtaining a low-dimensional time sequence feature vector sequence of each time sequence segment, and forming a total sequence of tactile feature vectors from the low-dimensional time sequence feature vector sequences of all time sequence segments;

[0016] inputting the total sequence of tactile feature vectors into a fatigue state recognition model which is completed training, and outputting a fatigue state classification result of the driver from the fatigue state recognition model;

[0017] The fatigue state identification model includes an encoder, an attention generation module, and a decoder. The encoder includes a feature extraction unit and a dynamic triggering neuron. The dynamic triggering neuron converts the continuous value features obtained by the feature extraction unit into a pulse sequence through membrane potential accumulation, threshold discharge, and reset mechanisms.

[0018] The attention generation module converts the pulse sequence into pulse-form query signals, key signals, and value signals; by calculating the pulse correlation between the query signal and the key signal, the value signal is weighted and fused to obtain attention features; and the attention features are fused with the pulse sequence to obtain enhanced features.

[0019] The decoder consists of a linear layer, a dynamically triggered neuron, and a fully connected classification head. The linear layer and the dynamically triggered neuron process the enhanced features to obtain higher-order features. The higher-order features are concatenated with the enhanced features and input into the fully connected classification head for driving state detection, and the fatigue state classification result is output.

[0020] Preferably, the distribution of the pressure sensor array on the vehicle steering wheel is as follows: pressure sensors are respectively set at the left and right hand grip positions of the steering wheel;

[0021] The pressure sensor array on the vehicle seat is distributed as follows: pressure sensors are installed on both the seat cushion and the seat back.

[0022] Preferably, the temporal tactile signal is divided into multiple temporal segments using a sliding time window method.

[0023] Preferably, the method for feature dimensionality reduction for each time series segment employs Pearson analysis to calculate the Pearson correlation coefficient between any two statistical features. for:

[0024] ;

[0025] in, and Represent any two statistical characteristics; Indicate statistical characteristics The i-th data point, Indicate statistical characteristics Average of data points , Indicate statistical characteristics The i-th data point, Indicate statistical characteristics Average of data points ;

[0026] For any two statistical characteristics and When the Pearson correlation coefficient If the absolute value is greater than the preset value, the statistical feature is removed. and One of them is to achieve feature dimensionality reduction and obtain a low-dimensional time-series feature vector sequence.

[0027] Preferably, the encoder includes multiple channels, each channel including a feature extraction unit and a dynamic triggering neuron, and performs parallel processing on each low-dimensional temporal feature vector sequence in the total tactile feature vector sequence through multiple channels.

[0028] Preferably, the feature extraction unit includes a Conv layer, a ReLU layer, and a MaxPooling layer. The encoder's processing formula for converting the low-dimensional temporal feature vector sequence into a pulse sequence is as follows:

[0029] ;

[0030] in, Represents a pulse sequence. Indicates the Conv layer weights. The th element in the total sequence of tactile feature vectors Low-dimensional temporal feature vectors corresponding to each time segment This represents the convolution operation. Indicates nonlinear activation. This indicates a max pooling operation. This indicates that neurons are dynamically triggered.

[0031] Preferably, the dynamic triggering neuron converts the continuous value features acquired by the feature extraction unit into a pulse sequence in the following way:

[0032] Accumulation of membrane potential: ;

[0033] Threshold-based discharge: ;

[0034] Reset mechanism: ;

[0035] in, This represents the membrane potential at the current time step. This represents the membrane potential at the previous time step. Indicates the membrane potential threshold. This represents the input signal at the current time step, i.e., the feature vector at the current time step in the low-dimensional time-series feature vector sequence. For pulse discharge function, Indicates the pulse discharge function After determining the discharge output pulse, the membrane potential is reset to 0.

[0036] Preferably, the encoder further comprises position encoding on the pulse sequence after converting the haptic feature vector sequence into the pulse sequence, the position encoding The calculation formula is:

[0037] ;

[0038] wherein, is batch normalization, denotes a two-dimensional convolution operation;

[0039] The pulse sequence after position encoding is denoted as: .

[0040] Preferably, the processing procedure of the decoder on the enhanced feature is:

[0041] ;

[0042] ;

[0043] wherein, is the feature representation obtained after the enhanced feature is processed by the linear layer and the dynamic trigger neuron, is the enhanced feature, is the learnable weight of the linear layer of the decoder, is the classification output result, denotes a concatenation operation on and , and is a classification function.

[0044] Preferably, the loss function of the fatigue state recognition model is:

[0045] ;

[0046] wherein, denotes the label of the training sample, the fatigue state is valued as 1, and the normal state is valued as 0, denotes the number of training samples, is the probability that the fatigue state recognition model predicts the training sample as the fatigue state.

[0047] Compared with the prior art, the application can achieve the following beneficial effects:

[0048] ​The application provides a driving state detection method based on a dynamic trigger neuron, which not only improves the traditional scheme of monitoring the fatigue state based on vehicle motion information, physiological signals or driving images, but also proposes a fatigue state monitoring and recognition method based on tactile information, solves the problems of the traditional scheme, such as being easily disturbed by the environment, interfering with the driver, and complex sensing devices, and further solves the problem that the traditional fatigue state monitoring algorithm is difficult to adapt to a vehicle chip. The computing ability and computing precision of the vehicle chip are usually poor, and the traditional fatigue state monitoring algorithm has a very high requirement for the computing ability of the chip, so the traditional fatigue state monitoring algorithm can obtain good results on an experimental computer, but there are many difficulties in the actual operation of the vehicle chip. In order to solve the existing problems, the application proposes a statistical feature parameterization method to calculate a plurality of core indexes for the original tactile signal, combines a redundancy elimination strategy based on a Pearson correlation coefficient to compress the feature dimension, eliminates data redundancy while retaining the strong correlation between the driver's behavior and physiological state, and based on a membrane potential accumulation-discharge model of a dynamic trigger neuron (MDTN), converts the feature input into a signal in the form of a pulse for information transmission, the dynamic trigger mechanism ensures that the signal transmission is activated when a high information event is triggered, that is, the signal energy is transmitted only when the threshold is reached, which ensures the transmission of key information while reducing the energy consumption of the invalid period, and generates a pulse type relative position embedding through convolution and the dynamic trigger neuron, constructs a spatial correlation modeling of the tactile signal, retains the spatial relationship of the sensor, that is, retains the signal relationship of the tactile signals at different positions, and establishes a pulse signal attention mechanism in the attention weight calculation process, converts the query (Q), key (K) and value (V) vectors into 0 / 1 pulse sequences, and replaces the floating point Softmax operation with pulse counting, which adapts to the low-precision computing unit of the vehicle chip. BRIEF DESCRIPTION OF DRAWINGS

[0049] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of the application illustrated in the drawings, and their description, are presented to add generic scope to the application. In the drawings:

[0050] Figure 1 It is a technical roadmap of the driving state detection based on the dynamic trigger neuron according to the embodiments of the application;

[0051] Figure 2 It is a sensor distribution schematic diagram of the tactile signal acquisition system according to the embodiments of the application;

[0052] Figure 3 It is a left hand grip parameter-time curve diagram of the driver in the normal state and the fatigue state according to the embodiments of the application;

[0053] Figure 4A right hand grip force parameter-time curve of a driver in a normal state and a fatigue state is provided according to an embodiment of the present application;

[0054] Figure 5 A sitting posture tactile parameter-time curve of a driver in a normal state and a fatigue state is provided according to an embodiment of the present application;

[0055] Figure 6 A Pearson correlation coefficient statistical diagram of statistical features is provided according to an embodiment of the present application;

[0056] Figure 7 An architecture diagram of a fatigue state recognition model is provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below with reference to the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and do not constitute a limitation on the present application. In different embodiments, similar elements are associated with similar element labels. In the following embodiments, many details are described in order to better understand the present application. However, those skilled in the art can easily recognize that some features can be omitted in different cases, or can be replaced by other elements, materials or methods. In some cases, some operations related to the present application are not shown or described in the specification in order to avoid the core part of the present application being overwhelmed by too much description, and it is not necessary to describe these related operations in detail for those skilled in the art based on the description in the specification and general technical knowledge in the art.

[0058] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other to form various embodiments without conflict. Meanwhile, the steps or actions in the method description can also be sequentially adjusted or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and drawings are only for the purpose of clearly describing a certain embodiment, and do not mean a necessary sequence, unless otherwise stated that a certain sequence must be followed.

[0059] In the description of the present application, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second" and the like are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined with "first", "second" and the like can be explicitly or implicitly included one or more. In the description of the present application, unless otherwise stated, the meaning of "a plurality of" is two or more.

[0060] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be connected inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood through specific circumstances.

[0061] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0062] Please refer to Figure 1 In an embodiment of the present application, a driving state detection method based on dynamic trigger neurons is provided, tactile information is introduced as a data source for driving state detection, the complex characteristics of driver muscle fatigue degree and operation stability decay are synchronously captured through the space-time distribution of steering wheel holding pressure and seat contact force, the precise mapping relationship between tactile information and driving state is constructed, and the fatigue state recognition model is innovatively designed. Through feature dimension reduction and feature pulsing, the calculation and energy consumption of invalid period are reduced while ensuring key information transmission, so that the fatigue state recognition model can run smoothly on the vehicle chip. The driving state detection method based on dynamic trigger neurons provided in the embodiment of the present application specifically includes the following steps:

[0063] Obtain the time sequence tactile signal collected by the pressure sensor array on the steering wheel and seat of the vehicle;

[0064] The time sequence tactile signal is divided into multiple time sequence segments, feature statistics and dimension reduction are performed on each time sequence segment, a low-dimensional time sequence feature vector sequence of each time sequence segment is obtained, and all low-dimensional time sequence feature vector sequences of the time sequence segments form a total sequence of tactile feature vectors;

[0065] The total sequence of tactile feature vectors is input into the fatigue state recognition model that is completed training, and the fatigue state recognition model outputs a fatigue state classification result of the driver;

[0066] The fatigue state recognition model comprises an encoder, an attention generation module and a decoder, the encoder comprises a feature extraction unit and a dynamic trigger neuron, and the dynamic trigger neuron converts continuous value features obtained by the feature extraction unit into a pulse sequence through a membrane potential accumulation, threshold discharge and reset mechanism.

[0067] The attention generation module converts the pulse sequence into a query signal, a key signal and a value signal in the form of pulses, weights and fuses the value signal by calculating the pulse correlation between the query signal and the key signal to obtain an attention feature, and fuses the attention feature and the pulse sequence to obtain an enhanced feature.

[0068] The decoder comprises a linear layer, a dynamic trigger neuron and a fully connected classification head, processes the enhanced feature through the linear layer and the dynamic trigger neuron to obtain a high-order feature, splices the high-order feature and the enhanced feature into the fully connected classification head for driving state detection, and outputs the fatigue state classification result.

[0069] Specifically, first, a tactile signal collection system needs to be installed on the vehicle, the tactile signal collection system comprises a steering wheel tactile perception sub-module and a seat tactile perception sub-module, and a power supply for supplying power to the steering wheel tactile perception sub-module and the seat tactile perception sub-module.

[0070] As shown in FIG. Figure 2 The steering wheel tactile perception sub-module comprises pressure sensors arranged at left and right hand holding positions of the steering wheel. The specific distribution mode is that three pressure sensors are arranged at 9 o'clock and 3 o'clock directions of the steering wheel, each pressure sensor contains two sensitive units, and each sensitive unit can obtain one pressure signal. The steering wheel tactile perception sub-module is provided with a total of six pressure sensors, and can collect and obtain twelve time sequence tactile signals of the driver's hands.

[0071] The seat tactile perception sub-module includes pressure sensors arranged on the seat cushion and pressure sensors arranged on the seat back. Among them, the pressure sensors on the cushion and the back are in a 3*3 distribution form, and the pressure sensors in the seat tactile perception sub-module are all flexible tactile sensors including a single sensitive unit. The seat tactile perception sub-module is provided with 18 pressure sensors, which can collect 18 time sequence tactile signals related to the sitting posture of the driver. The tactile signal acquisition system obtains 30 pressure signals, which together form the time sequence tactile signal. The 30 pressure signals can be divided into four groups according to the position, namely the left hand, the right hand, the cushion and the back.

[0072] In order to facilitate subsequent construction of the mapping relationship between the tactile signal and the driving state through deep learning, it is necessary to verify the correlation between the tactile signal and the driving state. In order to simplify the number of input parameters of the subsequent fatigue state recognition model and improve the efficiency of the model, the 30 pressure signals are converted in the embodiment of the application.

[0073] Specifically, the 12 pressure signals of the left hand and the right hand are converted into the normal grip force mean and the normal grip force standard deviation, and the tangential grip force, including the left hand normal grip force mean NLM, the right hand normal grip force mean NRM, the left hand normal grip force standard deviation NLS, the right hand normal grip force standard deviation NRS, the left hand tangential grip force TL and the right hand tangential grip force TR.

[0074] The 9 pressure signals of the seat cushion are converted into the pressure mean SCM and the pressure standard deviation SCS.

[0075] The 9 pressure signals of the seat back are converted into the pressure mean SBM and the pressure standard deviation SBS.

[0076] In summary, the 30 pressure signals are converted into the above-mentioned 10 tactile feature parameters, and the 10 tactile feature parameters are used for driver state monitoring. The calculation formula of the 10 tactile feature parameters is as follows:

[0077] ;

[0078] Among them, The left hand tactile signal of any sensor n is represented as

[0079] ;

[0080] Among them, The right hand tactile signal of any sensor n is represented as

[0081] ;

[0082] ;

[0083] ;

[0084] wherein, L1-L6 are left-hand 6-way driver hand movement mechanics signals, L1, L3 and L5 can perceive the left-hand steering force component in the counterclockwise direction, and L2, L4 and L6 can perceive the left-hand steering force component in the clockwise direction;

[0085] ;

[0086] wherein, R1-R6 are right-hand 6-way driver hand movement mechanics signals, R1, R3 and R5 can perceive the right-hand steering force component in the counterclockwise direction, and R2, R4 and L6 can perceive the right-hand steering force component in the clockwise direction;

[0087] ;

[0088] wherein, represents the seat cushion pressure signal of any sensor n;

[0089]

[0090] ;

[0091] wherein, represents the seat back pressure signal of any sensor n;

[0092] ;

[0093] is a 9-way signal output by a pressure sensor array on the seat cushion, is a 9-way signal output by a pressure sensor array on the seat back.

[0094] In order to facilitate subsequent construction of a tactile signal and driving state mapping relationship through deep learning, it is necessary to verify the correlation between the tactile signal and the driving state. In order to verify and study whether the 10 tactile characteristic parameters are internally related to the driving state of the driver, the embodiment of the present application collects time sequence tactile signals under normal driving conditions and fatigue driving conditions through a tactile signal collection system. By comparing the time sequence tactile signals under the two conditions, it is verified that the 10 tactile characteristic parameters have a mapping relationship with the driving state of the driver. Among them, the normal driving condition is selected at 10 o'clock in the morning after the driver has a good rest, and the fatigue driving condition is selected at 2 o'clock in the afternoon. The experimental results obtained are shown in Figure 3 、 Figure 4 and Figure 5 .

[0095] wherein, Figure 3 The two graphs in the figure respectively represent the left-hand parameter curves of the driver in the normal state and the fatigue state over time.Figure 4 The two graphs in the image represent the changes of the right-hand parameter over time for drivers in normal and fatigued states, respectively.

[0096] From the steering wheel tactile parameter curves, TL and TR reflect the driver's behavioral characteristics when turning the steering wheel, NLM and NRM reflect the driver's grip strength, and NLS and NRS represent the uniformity of the driver's grip strength distribution. Under normal driving conditions, the driver's normal grip strength parameters (NLM, NRM, NLS, and NRS) remain relatively stable without significant fluctuations, while TL and TR show small-amplitude, high-frequency fluctuations, indicating that the driver is making small steering wheel turns, consistent with typical characteristics of normal driving. When fatigue sets in, the driver's grip strength on the steering wheel weakens, and they may be unable to maintain a standard grip posture. As can be seen from the graph, the fluctuation amplitude of TL and TR increases significantly, and the fluctuations become more irregular, reflecting unstable driving behavior under fatigue.

[0097] Figure 5 The two graphs in the figure represent the temporal changes in the seated tactile parameters of a driver in a normal and fatigued state, respectively. From the perspective of seated tactile parameters, SCM and SBM reflect the shift of the driver's center of gravity in the fore-and-aft direction, while SCS and SBS represent the degree of standardization of the driver's posture. Under normal driving conditions, the driver's posture changes are relatively small, and SCM shows an upward trend, indicating that the driver gradually shifts their center of gravity backward to adapt to the driving task. After entering a fatigued state, the driver's posture changes significantly increase, and the fluctuation amplitude and frequency of all four parameters increase significantly, indicating that posture changes are more frequent under fatigue. The overall trend shows that SBM gradually increases and SBS gradually decreases, indicating two points: First, the driver's control of the vehicle (such as turning the steering wheel, braking, and acceleration) becomes less smooth than during normal driving, and the range of motion increases; second, the driver relies more on the seat back, shifting their center of gravity backward, and reducing the weight borne by their feet.

[0098] As can be seen from the above, the 10 tactile parameters selected by this invention are internally related to the driver's state. That is, the mapping relationship between the driving state and the temporal tactile signal can be constructed through these 10 tactile feature parameters.

[0099] The temporal tactile signal acquired by the tactile signal acquisition system is a data stream. This data stream is divided into multiple temporal segments using a sliding time window method. Specifically, a fixed sliding step size is set, and the data within this step window is considered a temporal segment. Assume the total duration of the acquired temporal tactile signal is... The sliding step size is This data stream can then be divided into... A time sequence segment, specifically, the window size is set to 6s, and the sampling frequency in the window is 200Hz. Therefore, the time sequence tactile signal collected in a period of time can be expressed as:

[0100] ;

[0101] wherein, The feature vector sequence in an arbitrary time sequence segment or time window can be expressed as, Each time step in the corresponding time window includes a set of tactile time sequence features, and the set includes selected specific features in the time sequence segment.

[0102] To improve the driving state recognition capability, the embodiment of the present application also needs to further standardize the time sequence tactile signal, and count the features of the time sequence tactile signal corresponding to each time sequence segment. In the embodiment of the present application, the features counted in the time sequence segment include:

[0103] The mean value NLMM of the mean value of the left hand normal grip force in the time window;

[0104] The standard deviation NLMS of the mean value of the left hand normal grip force in the time window;

[0105] The mean value NLSM of the standard deviation of the left hand normal grip force in the time window;

[0106] The standard deviation NLSS of the standard deviation of the left hand normal grip force in the time window;

[0107] The mean value NRMM of the mean value of the right hand normal grip force in the time window;

[0108] The standard deviation NRMS of the mean value of the right hand normal grip force in the time window;

[0109] The mean value NRSM of the standard deviation of the right hand normal grip force in the time window;

[0110] The standard deviation NRSS of the standard deviation of the right hand normal grip force in the time window;

[0111] The standard deviation TLS of the left hand tangential grip force in the time window;

[0112] The standard deviation TRS of the right hand tangential grip force in the time window;

[0113] The mean value SCMM of the mean value of the seat cushion pressure in the time window;

[0114] The standard deviation SCMS of the mean value of the seat cushion pressure in the time window;

[0115] The mean value SCSM of the standard deviation of the seat cushion pressure in the time window;

[0116] The standard deviation of the standard deviation of seat cushion pressure (SCSS) within the time window;

[0117] The mean value of seat back pressure within the time window (SBMM);

[0118] Standard deviation of mean seat back pressure (SBMS) within the time window;

[0119] The mean of the standard deviation of seat back pressure (SBSM) within the time window;

[0120] The standard deviation of the standard deviation of seat back pressure (SBSS) within the time window.

[0121] Here, we will further explain using NLMM as an example. It is the average value of the left-hand normal grip force mean NLM over all sampling points within a 6-second time window. Each sampling point actually includes the pressure signals output by the 6 sensors corresponding to the left hand. The meanings of other parameters are similar and will not be elaborated here.

[0122] In this embodiment of the invention, 18 statistical features were selected. To reduce data redundancy caused by strongly correlated features, improve classification accuracy, reduce computational load, and avoid invalid calculations of duplicate data, this embodiment of the invention further preprocesses the hand feature vector and sitting posture feature vector, performs Pearson correlation analysis on these 18 tactile feature vectors, and removes redundant data. The specific processing procedure is as follows:

[0123] The Pearson analysis method was used to perform correlation analysis between hand feature vectors and sitting posture feature vectors. The Pearson analysis formula is as follows:

[0124] ;

[0125] in, and Represent any two statistical characteristics; Indicate statistical characteristics The i-th data point, Indicate statistical characteristics Average of data points , Indicate statistical characteristics The i-th data point, Indicate statistical characteristics Average of data points ;

[0126] For any two statistical characteristics and When the Pearson correlation coefficient If the absolute value is greater than the preset value, the statistical feature is removed. and one of the following, to realize feature dimension reduction, and obtain a low-dimensional time sequence feature vector sequence.

[0127] The structure of Pearson correlation analysis of 18 tactile feature vectors is shown in Figure 6 In the embodiment of the present application, the preset value is 0.5, and according to the analysis calculation result, only one feature with a correlation greater than 0.5 is retained, so that NLSM, TRS, SBMS, SCSM, SBSS and SCSS are removed. After dimension reduction, there are 12 features in total, namely NLMM, NLMS, NRMM, NRMS, NLSS, NRSM, NRSS, TLS, SCMM, SCMS, SBMM and SBSM. The 12 vectors constitute a low-dimensional time sequence feature vector sequence, that is, for any time window, the feature vector sequence will obtain a feature set at any time step t in , which can be represented as:

[0128] ;

[0129] wherein, respectively represent the values of the statistical features NLMM, NLMS, NRMM, NRMS, NLSS, NRSM, NRSS, TLS, SCMM, SCMS, SBMM and SBSM obtained by collecting data at any time step t in the feature vector sequence .

[0130] Thus, for all time sequence segments of the time sequence tactile signal collected in a period of time, the total sequence of tactile feature vectors formed by the low-dimensional time sequence feature vector sequence is .

[0131] To better represent the relationship between the left-hand tactile on the steering wheel, the right-hand tactile and the seat tactile, the tactile feature information at each time step is converted into the following three forms, to improve the feature extraction capability:

[0132] ;

[0133] ;

[0134] .

[0135] After data processing and analysis are completed, and before constructing the fatigue state identification model, a training sample dataset needs to be built. In this embodiment of the invention, the training samples are time-series tactile signals collected under normal driving conditions and fatigue driving conditions during the study of the correlation between tactile signals and driving state. This experimental data needs to be labeled, and the driver's state needs to be scored using the KSS sleepiness scale to assess the driver's fatigue driving level, generating data with true labels for normal or fatigued states. This training dataset is used for subsequent model training.

[0136] Further construct a fatigue state identification model, such as Figure 7 As shown, in this embodiment of the invention, the fatigue state identification model adopts an encoder-decoder architecture, including: an encoder, an attention generation module, and a decoder. The encoder is used to dynamically segment the input tactile feature vector sequence using trigger signals. The encoder includes multiple channels for synchronously processing temporal segments within each time window in parallel. Each channel of the encoder includes: a feature extraction unit and a dynamic trigger neuron. The feature extraction unit sequentially includes a Conv layer, a ReLU layer, and a MaxPooling layer. For any feature vector sequence within a time window... First, convolutional scanning is performed through the Conv layer to capture local spatiotemporal features in the data. Then, nonlinear activation is performed through the ReLU layer to introduce nonlinearity into the system, enabling the network to fit various complex mapping relationships. Finally, feature compression and enhancement are performed through the MaxPooling layer. By retaining the most significant features, the model becomes less sensitive to small displacements and changes in the input data, and the amount of data and subsequent computational complexity are reduced.

[0137] Dynamically triggered neurons are used to convert continuous-value features acquired by the feature extraction unit into pulse sequences. This pulse design differs from existing schemes in that it determines when to generate a pulse based on the intensity of the input features, through membrane potential accumulation, threshold discharge, and a reset mechanism. This pulses the continuous-value features for subsequent recognition, classification, and other tasks. The dynamically triggered neurons convert continuous-value features acquired by the feature extraction unit into pulse sequences in the following way:

[0138] Accumulation of membrane potential: ;

[0139] Threshold-based discharge: ;

[0140] Reset mechanism: ;

[0141] in, This represents the membrane potential at the current time step. This represents the membrane potential at the previous time step. Indicates the membrane potential threshold. This represents the input signal at the current time step, i.e., the feature vector at the current time step in the low-dimensional time-series feature vector sequence. The membrane potential accumulation function, For pulse discharge function, Indicates the pulse discharge function After determining the discharge output pulse, the membrane potential is reset to 0.

[0142] The accumulation of membrane potential is essentially a dynamic triggering process by which neurons process the currently received signal. The generated excitation level (membrane potential) is added to the excitation level generated by the previous input signal to form a total excitation level used to determine whether to trigger a pulse. The threshold-based discharge process occurs when the accumulated excitation level exceeds a critical point (membrane potential threshold). The activation generates a pulse, and finally the excitability after the output pulse is reset through a reset mechanism.

[0143] The encoder's data processing procedure is as follows:

[0144] ;

[0145] in, Represents a pulse sequence. Indicates the Conv layer weights. The th element in the total sequence of tactile feature vectors Low-dimensional temporal feature vectors corresponding to each time segment This represents the convolution operation. Indicates nonlinear activation. This indicates a max pooling operation. This indicates that neurons are dynamically triggered.

[0146] After dynamic triggering segmentation by the encoder, the pulse sequence corresponding to each time segment Its physical meaning is to map it to a D-dimensional pulse-form feature vector and divide it into N pulse-form patch sequences, where This indicates the length of the input time window for driver state identification. To effectively obtain global information, position encoding of the pulse sequence is also required; however, floating-point position encoding cannot accurately represent the input time window. Therefore, a conditional position embedding generator is used to generate relative positions in the form of pulses for embedding, and position encoding is performed. Location coding The formula for calculation is:

[0147] ;

[0148] in, batch normalization, denotes a two-dimensional convolution operation;

[0149] After the pulse sequence is marked and position encoded, the input sequence in each time window is represented by the sequence segmented by the dynamic trigger signal as follows: .

[0150] The sequence segmented by the dynamic trigger signal is input to the trigger signal attention generation module. Since the floating-point matrix multiplication in the traditional attention mechanism includes exponential calculation and division operation, it cannot be used directly. Therefore, in the embodiment of the present application, the pulse sequence after the marked position encoding is input to the attention generation module, which needs to ensure that the attention mechanism is still implemented in the pulse domain to avoid the energy-consuming floating-point operation in the traditional attention mechanism. Specifically, first, the pulse sequence after the marked position encoding is multiplied by three learnable weight matrices respectively, and feature recognition is performed from different dimensions, then batch normalization is performed to ensure data distribution consistency, and finally floating-point data is converted back to pulse sequence through dynamic trigger neurons to ensure that the Q, K and V vectors are still pulse signals containing only 0 and 1, and the Q, K and V vectors in pulse form are generated as follows:

[0151] ;

[0152] ;

[0153] ;

[0154] wherein, , and denote the query vector, the key vector and the value vector of the i-th time window, , , and denote the learnable weights for calculating the query vector, the key vector and the value vector.

[0155] In the traditional attention, this is floating-point matrix multiplication, while in the embodiment of the present application, the Q, K and V vectors are all in the pulse domain, so the complex floating-point operation is replaced by the pulse counting mechanism, and the value vector V is weighted by the pulse counting result, and the weight value is adjusted by the scaling factor, and then the pulse is converted again by the dynamic trigger neuron.

[0156] Specifically, after generating the Q, K and V vectors in pulse form, first, scaling fusion is performed through the scaling layer Scale, and then pulse constraint is performed through the dynamic trigger neuron MDTN, so that the attention weight obtained by the trigger signal attention generation module is as follows:

[0157] ;

[0158] wherein, is a scaling factor.

[0159] Further linear transformation is performed by a linear layer (Linear layer) and pulse constraint is performed by a dynamic trigger neuron MDTN, to obtain attention-enhanced features is:

[0160] ;

[0161] wherein, represents the learnable weight corresponding to the linear layer after attention output.

[0162] The final trigger signal attention generation module splices the original features and the attention-enhanced features by a Concat layer to obtain enhanced features that fuse global context information is:

[0163] ;

[0164] wherein, .

[0165] The enhanced features are input into the decoder, and the decoder module includes a linear layer, a dynamic trigger neuron, a Concat layer (feature splicing layer), and a fully connected classification head. For the input enhanced features , first, the linear layer and the dynamic trigger neuron of the decoder are processed to obtain:

[0166] ;

[0167] wherein, is the feature after linear transformation and pulse coding processing, is the learnable weight of the decoder linear layer.

[0168] Further, the is spliced with the input enhanced features by a Concat layer to retain feature information at different levels, and finally a fully connected classification head is used for classification output. The fully connected classification head outputs a classification result according to the predicted fatigue state probability value:

[0169] ;

[0170] wherein, is the classification output result, is the classification output result, represents the splicing operation on and , The classification function.

[0171] After the fatigue state recognition model is constructed, the model is trained by using the pre-labeled training sample data set. During the training process, the loss function of the fatigue state recognition model is constructed

[0172]

[0173] wherein, y represents the label of the training sample, the fatigue state takes the value 1, and the normal state takes the value 0, N represents the number of training samples, P represents the probability value of the fatigue state recognition model predicting the training sample as the fatigue state. If it is required to divide the driving state into multiple (more than two) categories, the loss function can be set as a multi-classification cross-entropy loss. The training weight in the model is updated by using an ode solver to reduce the value of the loss function, so as to obtain a model meeting the requirements. After the training is completed, the fatigue state recognition model can be deployed, and the pressure sensor array arranged on the steering wheel and the seat of the vehicle can be combined to realize the tactile data acquisition and the fatigue state monitoring of the driver. The present application not only realizes the non-invasive data acquisition, but also innovatively designs the monitoring model. The dynamic trigger neuron converts the continuous tactile features into a sparse pulse signal, which improves the overall state monitoring and recognition accuracy, reduces the overall computing energy consumption, and solves the problem that the existing algorithm cannot stably run on the vehicle chip.

[0174] In summary, the above only describes the preferred embodiments of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0175] The system, device, module or unit described in one or more embodiments above can be specifically implemented by a computer chip or entity, or by a product with certain functions. A typical implementation device is a computer. Specifically, the computer may, for example, be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0176]

[0177] ​​​It is also to be noted that the terms "comprising", "including", and any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0178] The various embodiments in the specification are described in progressive manner, and the same or similar parts among the various embodiments can be mutually referred to, and each embodiment focuses on the difference from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

Claims

1. A driving state detection method based on a dynamic trigger neuron, characterized in that, The method comprises: acquiring time-series tactile signals collected by a pressure sensor array on a vehicle steering wheel and a seat; dividing the time-series tactile signals into multiple time-series segments, performing feature statistics and dimension reduction on each time-series segment, and obtaining a low-dimensional time-series feature vector sequence of each time-series segment, and all low-dimensional time-series feature vector sequences of the time-series segments forming a total sequence of tactile feature vectors; inputting the total sequence of tactile feature vectors into a fatigue state recognition model that has been trained, and outputting a fatigue state classification result of a driver by the fatigue state recognition model; The fatigue state recognition model comprises an encoder, an attention generation module and a decoder, the encoder comprises a feature extraction unit and a dynamic trigger neuron, and the dynamic trigger neuron converts continuous value features obtained by the feature extraction unit into a pulse sequence through a membrane potential accumulation, threshold discharge and reset mechanism. The attention generation module converts the pulse sequence into a query signal, a key signal and a value signal in the form of pulses; the value signal is weighted and fused by calculating the pulse correlation between the query signal and the key signal to obtain an attention feature, and the attention feature and the pulse sequence are fused to obtain an enhanced feature; The decoder comprises a linear layer, a dynamic trigger neuron and a fully connected classification head, and a high-order feature is obtained by processing the enhanced feature through the linear layer and the dynamic trigger neuron, the high-order feature and the enhanced feature are spliced and input into the fully connected classification head for driving state detection, and a fatigue state classification result is output.

2. The driving state detection method based on a dynamic trigger neuron according to claim 1, characterized in that, The distribution mode of the pressure sensor array on the vehicle steering wheel is that pressure sensors are arranged at the left and right hand holding positions of the steering wheel respectively. The distribution mode of the pressure sensor array on the vehicle seat is that pressure sensors are arranged on the seat cushion and the seat back respectively. 3.The driving state detection method based on dynamic trigger neurons according to claim 1, characterized in that, The time-series tactile signals are divided into multiple time-series segments by a sliding time window method.

4. The driving state detection method based on a dynamic trigger neuron according to claim 3, characterized in that, The method of feature dimension reduction for each time sequence segment adopts Pearson analysis method to calculate Pearson correlation coefficient of any two statistical features is: ； in, and Represent any two statistical characteristics; Indicate statistical characteristics The i-th data point, Indicate statistical characteristics Average of data points , Indicate statistical characteristics The i-th data point, Indicate statistical characteristics Average of data points ; For any two statistical features and When the absolute value of the Pearson correlation coefficient is greater than a preset value, one of the statistical features and is removed, the feature dimensionality is reduced, and a low-dimensional time sequence feature vector sequence is obtained.

5. The driving state detection method based on a dynamic trigger neuron according to claim 1, characterized in that, The encoder comprises multiple channels, each channel comprising a feature extraction unit and a dynamic trigger neuron, and each low-dimensional time-series feature vector sequence in the total sequence of tactile feature vectors is processed in parallel by the multiple channels.

6. The driving state detection method based on a dynamic trigger neuron according to claim 5, characterized in that, The feature extraction unit comprises a Conv layer, a ReLU layer and a MaxPooling layer, and the processing formula of the encoder for converting the low-dimensional time-series feature vector sequence into a pulse sequence is: ； wherein, represents a pulse sequence, represents a Conv layer weight, is a low-dimensional temporal feature corresponding to the th time sequence segment in the total sequence of tactile feature vectors, represents a convolution operation, represents a nonlinear activation, represents a max-pooling operation, represents a dynamic trigger neuron.

7. The driving state detection method based on a dynamic trigger neuron according to claim 6, characterized in that, The dynamic trigger neuron converts the continuous value features obtained by the feature extraction unit into a pulse sequence in the following manner: Accumulation of membrane potential: ; Threshold-based discharge: ; Reset mechanism: ; wherein, represents the membrane potential at the current time step, represents the membrane potential at the previous time step, represents the membrane potential threshold, represents the input signal at the current time step, i.e. the feature vector at the current time step of the low-dimensional time series feature vector sequence, is a pulse discharge function, represents the reset of the membrane potential to 0 after the discharge output pulse, represents the reset of the membrane potential to 0 after the discharge output pulse.

8. The driving state detection method based on a dynamic trigger neuron according to claim 7, characterized in that, The encoder, after converting the haptic feature vector sequence into a pulse sequence, further comprises position encoding the pulse sequence, the position encoding The calculation formula is: ； wherein, is batch normalization, denotes a two-dimensional convolution operation; The pulse sequence marker position is encoded after the representation as: . 9.The driving state detection method based on dynamic trigger neurons according to claim 1, wherein, The processing process of the decoder for the enhanced feature is: ； ； wherein, is a feature representation of the augmented features after processing by a linear layer and a dynamic trigger neuron, is the augmented features, is a learnable weight of the linear layer of the decoder, is a classification output result, denotes applying a and concatenation operation, is a classification function.

10. The driving state detection method based on a dynamic trigger neuron according to claim 1, characterized in that, The loss function of the fatigue state recognition model is: ； wherein, represents a label of a training sample, the fatigue state is valued as 1, and the normal state is valued as 0, represents the number of training samples, is the probability that the fatigue state recognition model predicts the training sample to be the fatigue state.

Images