A driver electrocardio state monitoring classification system and method

Abstract

The application discloses a driver ECG state monitoring and classifying system and method. A microprocessor collects ECG signals at multiple time points through an ECG signal sensor, and artificially marks the real driving state category at each time point; a driving ECG state classifying neural network is constructed, and an optimized driving ECG state classifying neural network is obtained through optimization training; the ECG signal sensor collects real-time ECG signals and outputs the real-time ECG signals to the microprocessor, the microprocessor performs prediction and classification on the real-time ECG signals through the optimized driving ECG state classifying neural network, obtains a real-time driving state prediction category, and displays the real-time driving state prediction category through the display module; if the driving state prediction category belongs to driving fatigue, a voice warning is performed through a warning module. The application has the advantages that a light-weight deep learning model is adopted, the response speed is fast, and the functionality is strong; the physiological state of the driver can be monitored in real time, so that a warning can be given in the first time.

Description

Technical Field

[0001] This invention belongs to the field of intelligent driving technology, and in particular relates to a driver's electrocardiogram status monitoring and classification system and method. Background Technology

[0002] Currently, according to statistics from relevant departments, there are over 200,000 motor vehicle traffic accidents nationwide each year, and the driver's condition while driving is a major contributing factor. Therefore, monitoring the driver's condition during driving can analyze and provide early warnings, allowing the driver to rest or receive appropriate treatment, thereby preventing accidents. However, current physiological signal monitoring devices on the market, such as blood pressure monitors and electrocardiogram (ECG) acquisition devices, suffer from problems such as complex measurement operations, inconvenience in carrying, and delayed measurement results, making it difficult to achieve real-time monitoring of physiological status and provide assessment and early warning of the driver's health status while driving.

[0003] Currently, deep neural network models offer advantages such as fast analysis speed and good classification performance. However, due to the large amount of computation required, analysis is typically performed by uploading data to the cloud and then transmitting it back to the local device. This method cannot upload data when traversing areas with no signal, such as tunnels, and therefore cannot achieve real-time, long-term monitoring and analysis.

[0004] Edge computing devices refer to physical hardware located remotely at the edge of a network, possessing sufficient memory, processing power, and computing resources. Therefore, they are well-suited for processing acquired physiological signals and providing classification, diagnostic, and other processing functions. Furthermore, the car steering wheel, as one of the most frequently touched interior devices by the driver during driving, is suitable for various non-invasive physiological signal detection devices due to its prolonged contact with the driver.

[0005] An existing patent describes a multi-functional automotive steering wheel, comprising a microprocessor, a fingerprint sensor, a heart rate sensor, an arterial pressure sensor, a venous pressure sensor, a non-invasive blood glucose monitoring device, a temperature sensor, a wireless communication module, and a GPRS SMS communication device. It uses a wireless network to connect the multi-functional steering wheel to a cloud server. Multiple sensors are placed on the surface of the steering wheel where the driver grips it, allowing for real-time monitoring of bodily indicators. This method heavily relies on reliable data transmission for physiological signal processing, which in turn depends on the cloud server.

[0006] An existing patent describes a biometric automotive steering wheel, comprising a steering wheel body, a heart rate acquisition module, and a PC host computer. This biometric steering wheel utilizes a pulse oximeter sensor installed on the vehicle's steering wheel, along with a simulated front-end chip and microcontroller, to monitor the driver's key physiological states in real time. This allows for proactive hazard prediction, warnings, alarms, and reminders, thus assisting in safe driving and reducing the accident rate. However, this method only uses pulse signals as an indicator of the driver's physiological state. Summary of the Invention

[0007] To address the problem that existing vehicles cannot perform real-time, long-term monitoring of the driver's physiological state during driving and cannot intelligently assess and warn of risks such as sudden cardiac arrest, this invention proposes a driver electrocardiogram monitoring and classification system and method.

[0008] The technical solution of the present invention is a driver's electrocardiogram (ECG) status monitoring and classification system, comprising: a microprocessor, an ECG signal sensor, a display module, and an early warning module;

[0009] The microprocessor is connected in sequence to the electrocardiogram signal sensor, the display module, and the early warning module.

[0010] The microprocessor is positioned at the center of the vehicle's steering wheel; the electrocardiogram (ECG) signal sensor is positioned at the handgrip of the steering wheel; the display module is located in the center of the steering wheel; and the warning module is located to the right of the display module.

[0011] The electrocardiogram (ECG) signal sensor is used to acquire ECG signals in real time and output them to the microprocessor;

[0012] The microprocessor is used to classify the real-time acquired electrocardiogram signals into the driver's driving status category using the driver's electrocardiogram status classification method, output the classification to the display module for display, and issue an early warning through the early warning module.

[0013] The technical solution of this invention is a method for monitoring and classifying the electrocardiogram status of drivers, comprising the following steps:

[0014] Step 1: The microprocessor collects ECG signals at multiple moments using an ECG signal sensor, and manually labels the actual driving state category at each moment;

[0015] Step 2: Construct a driving ECG state classification neural network. Input the ECG signal at each moment into the driving ECG state classification neural network for prediction and classification to obtain the predicted driving state category at each moment. Combine the actual driving state category at each moment to construct a loss function model. Optimize and train the model using the Adam algorithm to obtain the optimized driving ECG state classification neural network.

[0016] Step 3: The ECG signal sensor collects real-time ECG signals and outputs them to the microprocessor. The microprocessor predicts and classifies the real-time ECG signals through an optimized driving ECG state classification neural network to obtain the real-time driving state prediction category and displays it through the display module. If the driving state prediction category belongs to driving fatigue, a voice warning is issued through the warning module.

[0017] Preferably, the driving ECG state classification neural network described in step 2 is composed of a first feature convolution module, a second feature convolution module, ..., a Kth feature convolution module and a linear classification module cascaded together in sequence;

[0018] The first feature convolution module is used to extract the first ECG feature from the ECG signal at each time moment and output it to the second feature convolution module.

[0019] The i-th feature convolution module is used to extract the i-th ECG feature from the (i-1)-th ECG feature and output it to the (i+1)-th feature convolution module; i∈[1,K];

[0020] The Kth feature convolution module is used to extract the Kth ECG feature from the (K-1)th ECG feature and output it to the linear classification module.

[0021] The linear classification module consists of a global pooling layer, a first linear layer, a second linear layer, and a third linear layer cascaded together.

[0022] The linear classification module is used to convert the Kth ECG feature into a driving state classification vector;

[0023] Each feature convolutional module consists of attention-based grouped convolutional modules and downsampling modules connected together;

[0024] The attention-based grouped convolution module comprises: an attention channel partitioning module, a first attention branch module, a second attention branch module, an attention module, and an attention channel rearrangement module;

[0025] The attention channel division module is connected to the first attention branch module and the second attention branch module respectively; the first attention branch module is connected to the attention module, and the second attention branch module is connected to the attention module; the attention module is connected to the attention channel rearrangement module.

[0026] The first attention branch module is composed of a first attention convolutional layer, an attention DW convolutional layer, and a second attention convolutional layer cascaded in sequence;

[0027] The second attention branch module is composed of a third attention convolutional layer and a fourth attention convolutional layer cascaded together;

[0028] The input to the attention channel segmentation module is the input to the i-th feature convolution module, defined as the i-th time series.

[0029] in, Let C represent the data value of the c-th channel at time t in the i-th feature convolutional module, where C is the total number of channels, T is the total number of time steps, and K is the total number of feature convolutional modules.

[0030] The attention channel segmentation module divides the input time series into:

[0031] The sub-pair sequence of the i-th feature convolutional module at time t. ;

[0032] The sub-odd sequence of the i-th feature convolutional module at time t ;

[0033] ;

[0034] in, Here are the channel indices for the sub-even sequence, and for channels 2, 4, ..., C of the original sequence. The channel index is the odd subsequence, and the channel index is the original sequence, which is channel 1, 3, ..., C-1.

[0035] The input to the first attention branch module is a sub-pair sequence, which is sequentially convolved through a first attention convolutional layer, an attention DW convolutional layer, and a second attention convolutional layer to obtain the value of channel c of the first attention branch module of the i-th feature convolutional module at time t.

[0036] in, The number of channels in the output sequence is half the number of input channels of the i-th feature convolutional module. T is the total number of time steps in the output sequence, which is the same as the total number of input time steps of the i-th feature convolutional module. K is the number of convolutional modules. i represents the output of the first attention branch module of the i-th feature convolutional module.

[0037] The input to the second attention branch module is a sub-odd sequence;

[0038] The sub-odd sequence is sequentially passed through the third and fourth attention convolutional layers for convolution operations, resulting in the value of channel c at time t in the second branch of the i-th feature convolutional module. ;

[0039] in, The number of channels in the output sequence is half the number of input channels in the i-th feature convolutional module; T is the total number of time steps in the output sequence, which is the same as the total number of input time steps in the i-th feature convolutional module; K is the number of convolutional channels; and i represents the output of the first attention branch module of the i-th feature convolutional module.

[0040] The input to the attention module is the output of the first attention branch module. The output of the second attention branch module The outputs of the two branches will first be concatenated to obtain the value of the branch output of channel c at time t of the i-th feature convolution module. ;

[0041] in,

[0042]

[0043] Where C is the number of channels in the output sequence, which is the same as the number of input channels in the i-th feature convolutional module; T is the total number of time steps in the output sequence, which is the same as the total number of input time steps in the i-th feature convolutional module; K is the number of convolutional modules; and i represents the i-th feature convolutional module.

[0044] The branch output of the i-th feature convolutional module is consistent with the number of channels and length of the initial time series;

[0045] The branch output of the i-th feature convolution module passes through the channel attention module to obtain the output value of channel c of the i-th feature convolution module at time t.

[0046] ;

[0047] in is the number of channels of the output with attention of the i-th feature convolutional module, which is the same as the number of input channels of the i-th feature convolutional module; T is the total number of time steps of the output with attention of the i-th feature convolutional module, which is the same as the total number of time steps of the input of the i-th feature convolutional module; K is the number of convolutional modules; and i represents the output of the attention module in the i-th feature convolutional module.

[0048] The input to the attention channel rearrangement module is The input will pass through an attention channel rearrangement module CS, which rearranges the channels [1,2,3,4…,C] into [1,3,5,…,C-1,2,4,…,C], resulting in the output value of channel c at time t of the i-th feature convolution module:

[0049] ;

[0050] in, The number of output channels of the i-th feature convolutional module is the same as the number of input channels of the i-th feature convolutional module; T is the total number of output times of the i-th feature convolutional module, which is the same as the total number of input times of the i-th feature convolutional module; K is the number of convolutional modules; and i represents the output of the i-th feature convolutional module.

[0051] The downsampling module comprises: a first downsampling branch module, a second downsampling branch module, and a downsampling channel rearrangement module;

[0052] The first downsampling branch module is connected to the downsampling channel rearrangement module; the second downsampling branch module is connected to the downsampling channel rearrangement module.

[0053] The first downsampling branch module is composed of a first downsampling convolutional layer, a DW downsampling convolutional layer, and a second downsampling convolutional layer cascaded together.

[0054] The second downsampling branch module consists of a cascaded downsampling max pooling layer and a third downsampling convolutional layer.

[0055] The nth channel time of the first downsampling branch module of the m-th downsampling module t The value is: .

[0056] Where N represents the number of channels, L represents the number of timestamps, and m represents the m-th downsampling module among M downsampling modules;

[0057] In the first downsampling branch module, The first downsampling convolutional layer, the DW downsampling convolutional layer, and the second downsampling convolutional layer are sequentially used to obtain the nth channel time of the first downsampling branch module of the m-th downsampling module. t The output value is:

[0058] ;

[0059] Where N represents the number of output channels of the first downsampling branch, which is the same as the number of input channels. This indicates the time number of the first downsampling branch, which is half of the input time number, and m represents the m-th downsampling module among the M downsampling modules;

[0060] The input to the second downsampling branch module is also a time series. ;

[0061] The nth channel time of the second downsampling branch of the m-th downsampling module is obtained by sequentially passing through a downsampling max pooling layer and a third downsampling convolutional layer, and then performing pooling and convolution operations respectively. l The output value is:

[0062]

[0063] Where N represents the number of output channels of the second downsampling branch, which is the same as the number of input channels. This indicates the time number of the second downsampling branch, which is half of the input time number, and m represents the m-th downsampling module among the M downsampling modules;

[0064] The inputs to the downsampling channel mixing module are: the output of the first downsampling branch module. The output of the second downsampling branch module ;

[0065] and By splicing and rearranging the channels through the downsampling channel mixing module, the channels [1,2,3,4…,2N] are rearranged into [1,3,5,…,2N-1,2,4,…2N], resulting in the output value at time l of the nth channel in the mth downsampling module:

[0066]

[0067] Where 2N represents the number of output channels for the m-th downsampling, which is twice the number of input channels. This represents the m-th downsampling time, which is half the number of input time points. m represents the m-th downsampling module among M downsampling modules.

[0068] The linear classification module is composed of a global pooling layer, a first linear layer, a second linear layer, and a third linear layer cascaded together in sequence.

[0069] The input value of the linear classification module at time t in the nth channel is:

[0070]

[0071] in, This represents the total number of channels output by the Kth downsampling module. This represents the total number of timestamps output by the Kth downsampling module.

[0072] Will As input to the global pooling layer, global pooling is performed at time n, and feature dimensionality reduction yields the numerical value of the global pooling layer in dimension n. ;in The dimension of the output is the same as the total number of channels output by the Kth downsampling module.

[0073] The output of the global pooling layer is linearly mapped sequentially through the first linear layer, the second linear layer, and the third linear layer to obtain a classification vector. ;

[0074] Where O represents the number of categories. This represents the value of the o-th category in the classification vector;

[0075] The loss function model described in step 2 is as follows:

[0076] The classification vector By comparing with real label vectors Calculate the loss function to optimize the network;

[0077] This represents the value of the o-th category in the label vector, when the categorized sample belongs to category o. If it is 1, otherwise, =0;

[0078] The label vector value for category J is:

[0079]

[0080] The loss function of the deep learning network model is the cross-entropy loss function, defined as follows:

[0081]

[0082]

[0083] in, To input the value of the electrocardiogram signal in category o, The softmax score is the score by which the current sample belongs to class o, and it is calculated as described above. This represents the value of the current sample at the o-th position in the classification vector.

[0084] The beneficial effects of this invention are: This invention adopts a lightweight deep learning model, which has a short training time and fast response, and can quickly process the driver's electrocardiogram signal, thereby realizing real-time monitoring of physiological state. Attached Figure Description

[0085] Figure 1 : Flowchart of the method according to an embodiment of the present invention;

[0086] Figure 2 : Block diagram of the neural network structure for classifying electrocardiogram states while driving according to an embodiment of the present invention;

[0087] Figure 3 : A block diagram of the linear classification module in an embodiment of the present invention;

[0088] Figure 4 : A block diagram of the attention-based grouped convolution module structure according to an embodiment of the present invention;

[0089] Figure 5 : A structural block diagram of the downsampling module according to an embodiment of the present invention; Detailed Implementation

[0090] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0091] In specific implementation, the method proposed in the technical solution of this invention can be automatically executed by those skilled in the art using computer software technology. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of this invention and computer equipment including the computer program running the corresponding computer program, should also be within the protection scope of this invention.

[0092] The technical solution of the system in the embodiment of the present invention is described below as a driver's electrocardiogram (ECG) status monitoring and classification system, including: a microprocessor, an ECG signal sensor, a display module, and an early warning module;

[0093] The microprocessor is connected in sequence to the electrocardiogram signal sensor, the display module, and the early warning module.

[0094] The microprocessor is positioned at the center of the vehicle's steering wheel; the electrocardiogram (ECG) signal sensor is positioned at the handgrip of the steering wheel; the display module is located in the center of the steering wheel; and the warning module is located to the right of the display module.

[0095] The electrocardiogram (ECG) signal sensor is used to acquire ECG signals in real time and output them to the microprocessor;

[0096] The microprocessor is used to classify the real-time acquired electrocardiogram signals into the driver's driving status category using the driver's electrocardiogram status classification method, output the classification to the display module for display, and issue an early warning through the early warning module.

[0097] The microprocessor selected is an Intel® Celeron® N3350 processor;

[0098] The selected electrocardiogram signal sensor is the AD8233 biopotential sensor;

[0099] The selected display module is a 128*32 dot matrix display screen;

[0100] The early warning module is selected as the SC110 buzzer;

[0101] The following is combined Figures 1 to 5 The technical solution of the method described in this invention is a driver's electrocardiogram (ECG) status monitoring and classification method, and the specific steps are as follows:

[0102] like Figure 1 The diagram shown is a flowchart of a method according to an embodiment of the present invention.

[0103] Step 1: The microprocessor collects ECG signals at multiple moments using an ECG signal sensor, and manually labels the actual driving state category at each moment;

[0104] Step 2: Construct a driving ECG state classification neural network. Input the ECG signal at each moment into the driving ECG state classification neural network for prediction and classification to obtain the predicted driving state category at each moment. Combine the actual driving state category at each moment to construct a loss function model. Optimize and train the model using the Adam algorithm to obtain the optimized driving ECG state classification neural network.

[0105] like Figure 2 As shown, the driving ECG state classification neural network described in step 2 is composed of a first feature convolution module, a second feature convolution module, ..., a Kth feature convolution module and a linear classification module cascaded together.

[0106] The first feature convolution module is used to extract the first ECG feature from the ECG signal at each time moment and output it to the second feature convolution module.

[0107] The i-th feature convolution module is used to extract the i-th ECG feature from the (i-1)-th ECG feature and output it to the (i+1)-th feature convolution module; i∈[1,K];

[0108] The Kth feature convolution module is used to extract the Kth ECG feature from the (K-1)th ECG feature and output it to the linear classification module, where K=5;

[0109] like Figure 3 As shown, the linear classification module consists of a global pooling layer, a first linear layer, a second linear layer, and a third linear layer cascaded together.

[0110] The linear classification module is used to convert the Kth ECG feature into a driving state classification vector;

[0111] Each feature convolutional module consists of attention-based grouped convolutional modules and downsampling modules connected together;

[0112] The attention-based grouped convolution module comprises: an attention channel partitioning module, a first attention branch module, a second attention branch module, an attention module, and an attention channel rearrangement module;

[0113] like Figure 4 As shown, the attention channel division module is connected to the first attention branch module and the second attention branch module respectively; the first attention branch module is connected to the attention module, and the second attention branch module is connected to the attention module; the attention module is connected to the attention channel rearrangement module.

[0114] The first attention branch module is composed of a first attention convolutional layer, an attention DW convolutional layer, and a second attention convolutional layer cascaded in sequence;

[0115] The second attention branch module is composed of a third attention convolutional layer and a fourth attention convolutional layer cascaded together;

[0116] The input to the attention channel segmentation module is the input to the i-th feature convolution module, defined as the i-th time series. ,in Specifically, when i=1, C=2, T=256; when i=5, C=32, T=16;

[0117] in, Let C represent the data value of the c-th channel at time t in the i-th feature convolutional module, where C is the total number of channels, T is the total number of time steps, and K is the total number of feature convolutional modules.

[0118] The attention channel segmentation module divides the input time series into:

[0119] The sub-pair sequence of the i-th feature convolutional module at time t. ;

[0120] The sub-odd sequence of the i-th feature convolutional module at time t

[0121] In particular, when i=1,

[0122] in, Here are the channel indices for the sub-even sequence, and for channels 2, 4, ..., C of the original sequence. The channel index is the odd subsequence, and the channel index is the original sequence, which is channel 1, 3, ..., C-1.

[0123] The input to the first attention branch module is a sub-pair sequence, which is sequentially convolved through a first attention convolutional layer, an attention DW convolutional layer, and a second attention convolutional layer to obtain the value of channel c of the first attention branch module of the i-th feature convolutional module at time t. in Specifically, when i=1, C / 2 = 1, T=256; when i=5, C=16, T=16;

[0124] in, The number of channels in the output sequence is half the number of input channels of the i-th feature convolutional module. T is the total number of time steps in the output sequence, which is the same as the total number of input time steps of the i-th feature convolutional module. K is the number of convolutional modules. i represents the output of the first attention branch module of the i-th feature convolutional module.

[0125] The input to the second attention branch module is a sub-odd sequence;

[0126] The sub-odd sequence is sequentially passed through the third and fourth attention convolutional layers for convolution operations, resulting in the value of channel c at time t in the second branch of the i-th feature convolutional module. .in In particular, when i=1, C / 2 =1, T=256; when i=5, C=16, T=16;

[0127] in, The number of channels in the output sequence is half the number of input channels in the i-th feature convolutional module; T is the total number of time steps in the output sequence, which is the same as the total number of input time steps in the i-th feature convolutional module; K is the number of convolutional channels; and i represents the output of the first attention branch module of the i-th feature convolutional module.

[0128] The input to the attention module is the output of the first attention branch module. The output of the second attention branch module The outputs of the two branches will first be concatenated to obtain the value of the branch output of channel c at time t of the i-th feature convolution module. ;.in Specifically, when i=1, C=2, T=256; when i=5, C=16, T=16

[0129] in,

[0130]

[0131] Specifically, when i=1, = ， =

[0132] Where C is the number of channels in the output sequence, which is the same as the number of input channels in the i-th feature convolutional module; T is the total number of time steps in the output sequence, which is the same as the total number of input time steps in the i-th feature convolutional module; K is the number of convolutional modules; and i represents the i-th feature convolutional module.

[0133] The branch output of the i-th feature convolutional module is consistent with the number of channels and length of the initial time series;

[0134] The branch output of the i-th feature convolution module passes through the channel attention module to obtain the output value of channel c of the i-th feature convolution module at time t.

[0135] .in In particular, when i=1, C=2, T=256; when i=5, C=16, T=16;

[0136] in is the number of channels of the output with attention of the i-th feature convolutional module, which is the same as the number of input channels of the i-th feature convolutional module; T is the total number of time steps of the output with attention of the i-th feature convolutional module, which is the same as the total number of time steps of the input of the i-th feature convolutional module; K is the number of convolutional modules; and i represents the output of the attention module in the i-th feature convolutional module.

[0137] The input to the attention channel rearrangement module is The input will pass through an attention channel rearrangement module CS, which rearranges the channels [1,2,3,4…,C] into [1,3,5,…,C-1,2,4,…,C], resulting in the output value of channel c at time t of the i-th feature convolution module:

[0138] .in In particular, when i=1, C=2, T=256; when i=5, C=16, T=16;

[0139] in, The number of output channels of the i-th feature convolutional module is the same as the number of input channels of the i-th feature convolutional module; T is the total number of output times of the i-th feature convolutional module, which is the same as the total number of input times of the i-th feature convolutional module; K is the number of convolutional modules; and i represents the output of the i-th feature convolutional module.

[0140] The downsampling module comprises: a first downsampling branch module, a second downsampling branch module, and a downsampling channel rearrangement module;

[0141] like Figure 5 As shown, the first downsampling branch module is connected to the downsampling channel rearrangement module; the second downsampling branch module is connected to the downsampling channel rearrangement module.

[0142] The first downsampling branch module is composed of a first downsampling convolutional layer, a DW downsampling convolutional layer, and a second downsampling convolutional layer cascaded together.

[0143] The second downsampling branch module consists of a cascaded downsampling max pooling layer and a third downsampling convolutional layer.

[0144] The nth channel time of the first downsampling branch module of the m-th downsampling module t The value is: Where M=5, In particular, when m=1, N=2 and T=256.

[0145] Where N represents the number of channels, L represents the number of timestamps, and m represents the m-th downsampling module among M downsampling modules;

[0146] In the first downsampling branch module, The first downsampling convolutional layer, the DW downsampling convolutional layer, and the second downsampling convolutional layer are sequentially used to obtain the nth channel time of the first downsampling branch module of the m-th downsampling module. t The output value is:

[0147] Where M=5, In particular, when m=1, N=2, T / 2=128.

[0148] Where N represents the number of output channels of the first downsampling branch, which is the same as the number of input channels. This indicates the time number of the first downsampling branch, which is half of the input time number, and m represents the m-th downsampling module among the M downsampling modules;

[0149] The input to the second downsampling branch module is also a time series. ;

[0150] The nth channel time of the second downsampling branch of the m-th downsampling module is obtained by sequentially passing through a downsampling max pooling layer and a third downsampling convolutional layer, and then performing pooling and convolution operations respectively. l The output value is:

[0151] Where M=5, In particular, when m=1, N=2, T / 2 =128.

[0152] Where N represents the number of output channels of the second downsampling branch, which is the same as the number of input channels. This indicates the time number of the second downsampling branch, which is half of the input time number, and m represents the m-th downsampling module among the M downsampling modules;

[0153] The inputs to the downsampling channel mixing module are: the output of the first downsampling branch module. The output of the second downsampling branch module ;

[0154] and By splicing and rearranging the channels through the downsampling channel mixing module, the channels [1,2,3,4…,2N] are rearranged into [1,3,5,…,2N-1,2,4,…2N], resulting in the output value at time l of the nth channel in the mth downsampling module:

[0155] Where M=5, In particular, when m=1, 2N=4, T / 2 =128.

[0156] Where 2N represents the number of output channels for the m-th downsampling, which is twice the number of input channels. This represents the m-th downsampling time, which is half the number of input time points. m represents the m-th downsampling module among M downsampling modules.

[0157] The linear classification module is composed of a global pooling layer, a first linear layer, a second linear layer, and a third linear layer cascaded together in sequence.

[0158] The input value of the linear classification module at time t in the nth channel is:

[0159]

[0160] in, This represents the total number of channels output by the Kth downsampling module. This represents the total number of timestamps output by the Kth downsampling module.

[0161] Will As input to the global pooling layer, global pooling is performed at time n, and feature dimensionality reduction yields the numerical value of the global pooling layer in dimension n. ;in The dimension of the output is 64, which is the same as the total number of channels output by the Kth downsampling module.

[0162] The output of the global pooling layer is linearly mapped sequentially through the first linear layer, the second linear layer, and the third linear layer to obtain a classification vector. ;

[0163] Where O represents the number of categories. This represents the value of the o-th category in the classification vector;

[0164] The loss function model described in step 2 is as follows:

[0165] The classification vector By comparing with real label vectors Calculate the loss function to optimize the network;

[0166] ;

[0167] The label vector value for category J is:

[0168]

[0169] The loss function of the deep learning network model is the cross-entropy loss function, defined as follows:

[0170]

[0171]

[0172] in, To input the value of the electrocardiogram signal in category o, The softmax score is the score by which the current sample belongs to class o, and it is calculated as described above. This represents the value of the current sample at the o-th position in the classification vector.

[0173] Step 3: The ECG signal sensor collects real-time ECG signals and outputs them to the microprocessor. The microprocessor predicts and classifies the real-time ECG signals through an optimized driving ECG state classification neural network to obtain the real-time driving state prediction category and displays it through the display module. If the driving state prediction category belongs to driving fatigue, a voice warning is issued through the warning module.

[0174] Although this invention uses terms such as microprocessor, ECG signal sensor, display module, and early warning module frequently, the possibility of using other terms is not excluded. These terms are used merely for the convenience of describing and explaining the essence of this invention; interpreting them as any additional limitation would contradict the spirit of this invention.

[0175] It should be understood that the above description of the preferred embodiments is quite detailed, but it should not be considered as a limitation on the scope of protection of this invention. Those skilled in the art, under the guidance of this invention, can make substitutions or modifications without departing from the scope of protection of the claims of this invention, and all such substitutions or modifications fall within the scope of protection of this invention. The scope of protection of this invention should be determined by the appended claims.

Claims

1. A driver's electrocardiogram (ECG) status monitoring and classification system, characterized in that, include: Microprocessor, ECG signal sensor, display module, and early warning module; The microprocessor is connected in sequence to the electrocardiogram signal sensor, the display module, and the early warning module. The microprocessor is positioned at the center of the vehicle's steering wheel; the electrocardiogram (ECG) signal sensor is positioned at the handgrip of the steering wheel; the display module is located in the center of the steering wheel; and the warning module is located to the right of the display module. The electrocardiogram (ECG) signal sensor is used to acquire ECG signals in real time and output them to the microprocessor; The microprocessor is used to classify the real-time acquired electrocardiogram signals into the driver's driving status category using the driver's electrocardiogram status monitoring and classification method, output the classification to the display module for display, and issue an early warning through the early warning module. The driver's electrocardiogram status monitoring and classification method includes the following steps: Step 1: The microprocessor collects ECG signals at multiple moments using an ECG signal sensor, and manually labels the actual driving state category at each moment; Step 2: Construct a driving ECG state classification neural network. Input the ECG signal at each moment into the driving ECG state classification neural network for prediction and classification to obtain the predicted driving state category at each moment. Combine the actual driving state category at each moment to construct a loss function model. Optimize and train the model using the Adam algorithm to obtain the optimized driving ECG state classification neural network. Step 3: The ECG signal sensor collects real-time ECG signals and outputs them to the microprocessor. The microprocessor predicts and classifies the real-time ECG signals through an optimized driving ECG state classification neural network to obtain the real-time driving state prediction category and displays it through the display module. If the driving state prediction category belongs to driving fatigue, a voice warning is issued through the warning module. The driving ECG state classification neural network described in step 2 is composed of a first feature convolution module, a second feature convolution module, ..., a Kth feature convolution module and a linear classification module cascaded together in sequence; The first feature convolution module is used to extract the first ECG feature from the ECG signal at each time moment and output it to the second feature convolution module. The i-th feature convolution module is used to extract the i-th ECG feature from the (i-1)-th ECG feature and output it to the (i+1)-th feature convolution module; i∈[1,K]; The Kth feature convolution module is used to extract the Kth ECG feature from the (K-1)th ECG feature and output it to the linear classification module. The linear classification module consists of a global pooling layer, a first linear layer, a second linear layer, and a third linear layer cascaded together. The linear classification module is used to convert the Kth ECG feature into a driving state classification vector; Each feature convolutional module consists of attention-based grouped convolutional modules and downsampling modules connected together; The attention-based grouped convolution module comprises: an attention channel partitioning module, a first attention branch module, a second attention branch module, an attention module, and an attention channel rearrangement module; The attention channel division module is connected to the first attention branch module and the second attention branch module respectively; the first attention branch module is connected to the attention module, and the second attention branch module is connected to the attention module; the attention module is connected to the attention channel rearrangement module. The first attention branch module is composed of a first attention convolutional layer, an attention DW convolutional layer, and a second attention convolutional layer cascaded in sequence; The second attention branch module is composed of a third attention convolutional layer and a fourth attention convolutional layer cascaded together; The input to the attention channel segmentation module is the input to the i-th feature convolution module, defined as the i-th time series. in, Let C represent the data value of the c-th channel at time t in the i-th feature convolutional module, where C is the total number of channels, T is the total number of time steps, and K is the total number of feature convolutional modules. The attention channel segmentation module divides the input time series into: The sub-pair sequence of the i-th feature convolutional module at time t. ; The sub-odd sequence of the i-th feature convolutional module at time t ; ； in, Here are the channel indices for the sub-even sequence, and for channels 2, 4, ..., C of the original sequence. The channel index is the odd subsequence, and the channel index is the original sequence, which is channel 1, 3, ..., C-1. The input to the first attention branch module is a sub-pair sequence, which is sequentially convolved through a first attention convolutional layer, an attention DW convolutional layer, and a second attention convolutional layer to obtain the value of channel c of the first attention branch module of the i-th feature convolutional module at time t. ; in, The number of channels in the output sequence is half the number of input channels of the i-th feature convolutional module. T is the total number of time steps in the output sequence, which is the same as the total number of input time steps of the i-th feature convolutional module. K is the number of convolutional modules. i represents the output of the first attention branch module of the i-th feature convolutional module. The input to the second attention branch module is a sub-odd sequence; The sub-odd sequence is sequentially passed through the third and fourth attention convolutional layers for convolution operations, resulting in the value of channel c at time t in the second branch of the i-th feature convolutional module. ; in, The number of channels in the output sequence is half the number of input channels in the i-th feature convolutional module; T is the total number of time steps in the output sequence, which is the same as the total number of input time steps in the i-th feature convolutional module; K is the number of convolutional channels; and i represents the output of the first attention branch module of the i-th feature convolutional module. The input to the attention module is the output of the first attention branch module. The output of the second attention branch module The outputs of the two branches will first be concatenated to obtain the value of the branch output of channel c at time t of the i-th feature convolution module. ; in, Where C is the number of channels in the output sequence, which is the same as the number of input channels in the i-th feature convolutional module; T is the total number of time steps in the output sequence, which is the same as the total number of input time steps in the i-th feature convolutional module; K is the number of convolutional modules; and i represents the i-th feature convolutional module. The branch output of the i-th feature convolutional module is consistent with the number of channels and length of the initial time series; The branch output of the i-th feature convolution module passes through the channel attention module to obtain the output value of channel c of the i-th feature convolution module at time t. ； in is the number of channels of the output with attention of the i-th feature convolutional module, which is the same as the number of input channels of the i-th feature convolutional module; T is the total number of time steps of the output with attention of the i-th feature convolutional module, which is the same as the total number of time steps of the input of the i-th feature convolutional module; K is the number of convolutional modules; and i represents the output of the attention module in the i-th feature convolutional module. The input to the attention channel rearrangement module is The input will pass through an attention channel rearrangement module CS, which rearranges the channels [1,2,3,4…,C] into [1,3,5,…,C-1,2,4,…,C], resulting in the output value of channel c at time t of the i-th feature convolution module: ； in, is the number of output channels of the i-th feature convolutional module, which is the same as the number of input channels of the i-th feature convolutional module; T is the total number of output times of the i-th feature convolutional module, which is the same as the total number of input times of the i-th feature convolutional module; K is the number of convolutional modules; and i represents the output of the i-th feature convolutional module.

2. The driver's electrocardiogram status monitoring and classification system according to claim 1, characterized in that: The downsampling module comprises: a first downsampling branch module, a second downsampling branch module, and a downsampling channel rearrangement module; The first downsampling branch module is connected to the downsampling channel rearrangement module; the second downsampling branch module is connected to the downsampling channel rearrangement module. The first downsampling branch module is composed of a first downsampling convolutional layer, a DW downsampling convolutional layer, and a second downsampling convolutional layer cascaded together. The second downsampling branch module consists of a cascaded downsampling max pooling layer and a third downsampling convolutional layer.

3. The driver electrocardiogram status monitoring and classification system according to claim 2, characterized in that: At time n of the first subsampling branch module of the m-th subsampling module. t The value is: ; Where N represents the number of channels, L represents the number of timestamps, and m represents the m-th downsampling module among M downsampling modules; In the first downsampling branch module, The first downsampling convolutional layer, the DW downsampling convolutional layer, and the second downsampling convolutional layer are sequentially used to obtain the nth channel time of the first downsampling branch module of the m-th downsampling module. t The output value is: ； Where N represents the number of output channels of the first downsampling branch, which is the same as the number of input channels. This indicates the time number of the first downsampling branch, which is half of the input time number, and m represents the m-th downsampling module among the M downsampling modules; The input to the second downsampling branch module is also a time series. ; The nth channel time of the second downsampling branch of the m-th downsampling module is obtained by sequentially passing through a downsampling max pooling layer and a third downsampling convolutional layer, and then performing pooling and convolution operations respectively. l The output value is: Where N represents the number of output channels of the second downsampling branch, which is the same as the number of input channels. This indicates the time number of the second downsampling branch, which is half of the input time number, and m represents the m-th downsampling module among the M downsampling modules; The inputs to the downsampling channel mixing module are: the output of the first downsampling branch module. The output of the second downsampling branch module ; and By splicing and rearranging the channels through the downsampling channel mixing module, the channels [1,2,3,4…,2N] are rearranged into [1,3,5,…,2N-1,2,4,…2N], resulting in the output value at time l of the nth channel in the mth downsampling module: Where 2N represents the number of output channels for the m-th downsampling, which is twice the number of input channels. This represents the m-th downsampling time, which is half the number of input time points. m represents the m-th downsampling module among M downsampling modules.

4. The driver's electrocardiogram status monitoring and classification system according to claim 3, characterized in that: The linear classification module is composed of a global pooling layer, a first linear layer, a second linear layer, and a third linear layer cascaded together in sequence. The input value of the linear classification module at time t in the nth channel is: in, This represents the total number of channels output by the Kth downsampling module. This represents the total number of timestamps output by the Kth downsampling module. Will As input to the global pooling layer, global pooling is performed at time n, and feature dimensionality reduction yields the numerical value of the global pooling layer in dimension n. ;in The dimension of the output is the same as the total number of channels output by the Kth downsampling module.

5. The driver's electrocardiogram status monitoring and classification system according to claim 4, characterized in that: The output of the global pooling layer is linearly mapped sequentially through the first linear layer, the second linear layer, and the third linear layer to obtain a classification vector. ; Where O represents the number of categories. This represents the value of the o-th category in the classification vector.

6. The driver electrocardiogram status monitoring and classification system according to claim 5, characterized in that: The loss function model described in step 2 is as follows: The classification vector By comparing with real label vectors Calculate the loss function to optimize the network; This represents the value of the o-th category in the label vector, when the categorized sample belongs to category o. If it is 1, otherwise, =0; The label vector value for category J is: The loss function is the cross-entropy loss function, defined as follows: in, For the sample's value in category o, The softmax score is the score by which the current sample belongs to class o, and it is calculated as described above. This represents the value of the current sample at the o-th position in the classification vector.

Images