Application Method and System of Electromyography-Based Intelligent Early Warning Watch for Fatigue Driving

Abstract

This invention discloses an application method and system for an electromyography (EMG)-based intelligent early warning watch for driver fatigue. The method includes: an inertial measurement module monitoring the user's posture; activating the working mode when the user is seated to obtain the rotation angle of the user's wrist; a signal acquisition module collecting EMG signals using patch electrodes; a camera module calculating the user's arm length; a signal processing module filtering the EMG signals; and a main control module extracting features from the EMG signals to calculate the angular velocity of the user's elbow joint. This invention accurately judges the user's state based on a temporal convolutional network and the user's elbow joint angular velocity; when the user is fatigued, the watch issues an early warning to remind the user and avoid dangerous events.

Description

Technical Field

[0001] This invention relates to the technical field of electromyography (EMG) watches, and more particularly to an application method and system for an EMG-based intelligent early warning watch for driver fatigue. Background Technology

[0002] Driver fatigue assessment is crucial for road safety, serving as a key step in preventing traffic accidents caused by drowsy driving. Statistics show that drowsy driving accounts for 15%-20% of all traffic accidents globally, and these accidents are often more severe. From a driving behavior perspective, fatigued drivers experience decreased attention, slower reaction times, impaired judgment, and may even experience brief periods of sleep, significantly increasing the risk of accidents. Therefore, designing a wearable device capable of assessing user fatigue and providing warnings of dangerous situations is essential.

[0003] Currently, some smart bracelets on the market use sensors for fatigue monitoring. For example, one smart bracelet contains 16 highly sensitive sensors, similar to those used in lie detectors, capable of accurately detecting the wearer's heart rate and skin conductivity data; by analyzing this data, it can determine whether the driver is fatigued. A smartwatch launched by a car manufacturer also has fatigue monitoring capabilities. It can monitor heart rate, blood oxygen, fatigue level, and sleep data 24 / 7. When the monitored data reaches a high-risk level, the watch will issue a warning.

[0004] Most of these wearable devices rely on electrocardiogram (ECG) signal characteristics to assess fatigue levels. ECG signals are susceptible to distortion due to various factors. For example, during driving, driver actions such as steering wheel operation and gear shifting can cause interference with muscle electrical signals, leading to deviations in key indicators like heart rate variability. In high-temperature environments, sweating reduces electrode conductivity; at high altitudes, hypoxia can cause compensatory increases in heart rate. Furthermore, improper device placement or a loose strap can directly affect signal acquisition quality. Summary of the Invention

[0005] To address the issue of insufficient accuracy in fatigue state assessment based on electrocardiogram (ECG) signal characteristics, this invention provides an application method and system for a warning watch that simultaneously utilizes electromyography (EMG) characteristics and elbow joint range of motion to assess fatigue state, effectively improving warning accuracy and preventing dangerous events.

[0006] The technical solution proposed in this invention is an application method for an electromyography-based intelligent early warning watch for driver fatigue, comprising the following steps:

[0007] S1: The inertial measurement module is responsible for monitoring the user's posture. When the user is in a seated position, the working mode is activated to measure the real-time acceleration of the user's wrist and calculate the movement speed and displacement of the user's wrist; the rotational angular velocity of the user's wrist is measured and the rotation angle of the user's wrist is obtained by combining the quaternion algorithm; the signal acquisition module starts to collect electromyographic signals using patch electrodes.

[0008] S2: The camera module is responsible for taking pictures of the user's arm, locating the user's elbow joint based on the pictures, and calculating the length of the user's arm.

[0009] S3: The signal processing module filters the acquired electromyographic signals to obtain filtered electromyographic signals;

[0010] S4: Feature extraction is performed on the filtered electromyography (EMG) signal using a temporal convolutional network; the main control module calculates the user's elbow joint angular velocity based on the user's wrist movement speed collected by the inertial measurement module and the user's arm length obtained by the camera module; the main control module triggers an early warning mechanism when the following two conditions are met simultaneously: first, the analysis result of the temporal convolutional network on the filtered EMG signal indicates a fatigue state; second, the calculated angular velocity of the user's elbow joint is greater than a preset threshold.

[0011] Optionally, S1 includes:

[0012] S11: The user wears the watch on their wrist, with the dial located at the center of the radiocarpal joint;

[0013] A dial coordinate system is established with the center of the radiocarpal joint as the origin. The plane passing through the center of the radiocarpal joint and parallel to the palm is defined as the xoz plane; the straight line perpendicular to the xoz plane and passing through the center of the radiocarpal joint is defined as the y-axis, with the positive direction of the y-axis pointing from the palm to the back of the hand; the direction parallel to the xoz plane, passing through the center of the radiocarpal joint, and from the wrist near the little finger to near the thumb is defined as the x-axis; and the direction parallel to the xoz plane, passing through the center of the radiocarpal joint, and pointing from the elbow to the fingers is defined as the z-axis.

[0014] S12: The inertial measurement module measures the real-time acceleration of the user's wrist movement, calculates the user's wrist movement speed and displacement, and combines the quaternion algorithm to obtain the rotation angle of the user's wrist.

[0015] S13: When the absolute value of the user's wrist movement displacement is less than the activity threshold and the absolute value of the user's wrist rotation angle is less than the standing threshold, the user is determined to be in a sitting position.

[0016] Optionally, S12 includes:

[0017] The velocity vector v(t) of the user's wrist at time t is expressed as:

[0018]

[0019] Where, v(t) = [v x (t),v y (t),v z (t)],v x (t),v y (t),v z (t) represents the user's wrist movement velocity in the x, y, and z directions at time t, respectively, and acce(τ) represents the user's wrist movement acceleration vector at time τ;

[0020] acce(τ)=[a x (τ),a y (τ),a z (τ)-pro(τ)·g]

[0021] Among them, a x (τ),a y (τ),a z (τ) represents the user's wrist acceleration in the x, y, and z directions at time τ, respectively, measured by the inertial measurement module; pro(τ) represents the gravitational acceleration to a. z The projection in the (τ) direction, where g represents the gravitational acceleration; at time 0, the user holds their arm vertically and initializes the watch, at which point pro(0) = 1;

[0022] The displacement vector dis(t) of the user's wrist at time t is expressed as:

[0023]

[0024] Where, dis(t) = [d x (t),d y (t),d z (t)],d x (t),d y (t),d z (t) represent the displacement of the user's wrist in the x, y, and z directions at time t, respectively;

[0025] When t = T, where T represents the time period, the absolute value of the user's wrist displacement, abdis, is represented as:

[0026] abdis=((d x (T)) 2 +(d y (T)) 2 +(d z (T)) 2 ) 1 / 2

[0027] Where, d x (T), d y (T), d z (T) represent the displacement of the user's wrist in the x, y, and z directions at time T, respectively;

[0028] At the initial moment, based on the initial attitude, the initial attitude quaternion q(0) is determined: q(0) = [1,0,0,0] T ,in[] T Represents the transpose matrix;

[0029] The angular velocity vector ω(t) of the user's wrist at time t is expressed as:

[0030] ω(t)=[ω x (t),ω y (t),ω z (t)]

[0031] Where, ω x (t),ω y (t),ω z (t) represent the angular velocities of the user's wrist in the x, y, and z directions at time t, respectively;

[0032] At time t, the attitude quaternion q(t) = pq ω (t),q x (t),q y (t),q z (t)] T , satisfying (q ω (t)) 2 +(q x (t)) 2 +(q y (t)) 2 +(q z (t)) 2 =1, where q ω (t) represents the real part of the attitude quaternion at time t, q x (t) represents the component of the imaginary part x of the attitude quaternion at time t, q y (t) represents the component of the imaginary part y of the attitude quaternion at time t, q z (t) represents the component of the imaginary part z of the attitude quaternion at time t, specifically:

[0033]

[0034] allang(t)=((ω x (t-1)+ω x (t)·Δt) 2 +(ω y (t-1)+ωy (t)·Δt) 2 +(ω z (t-1)+ω z (t)·Δt) 2 ) 1 / 2

[0035] Where allang(t) represents the rotation angle of the user's wrist at time t, ω x (t-1),ω y (t-1),ω z (t-1) represents the angular velocity of the user's wrist in the x, y, and z directions at time t-1, respectively, Δt represents the time sampling interval, sin is the sine function, and cos is the cosine function;

[0036] Instantaneous rate of change of attitude quaternion at time t Represented as:

[0037]

[0038] in, The quaternion ω represents the attitude quaternion q(t) at time t and the angular velocity at time t. q The instantaneous rate of change under the action of (t), This represents quaternion multiplication, where q(t+1) represents the attitude quaternion at time t+1, and ω... q (t) represents the quaternion representation of the angular velocity at time t, ω q (t)=[0,ω x (t),ω y (t),ω z (t)] T ;

[0039] When t = T, the angle of rotation of the user's wrist is expressed as:

[0040]

[0041] Where θ represents the pitch angle of the user's wrist rotation. ψ represents the roll angle of the user's wrist rotation, y represents the yaw angle of the user's wrist rotation, and q represents the roll angle of the user's wrist rotation. ω (T) represents the real part of the attitude quaternion at time T, q x (T) represents the component of the imaginary part x of the attitude quaternion at time T, q y (T) represents the component of the imaginary part y of the attitude quaternion at time T, q z (T) represents the component of the imaginary part z of the attitude quaternion at time T;

[0042] The absolute value of the user's wrist rotation angle, abangle, is represented as:

[0043]

[0044] Optionally, S1 includes:

[0045] The signal acquisition module includes patch electrodes and an adjustable wristband assembly;

[0046] The adjustable wristband assembly consists of multiple plastic plates connected by a high-elasticity elastic band. The plastic plates are in contact with the skin when worn, and patch electrodes are fixedly installed on their inner surfaces. The plastic plates are rigid or semi-rigid plastic plates.

[0047] The signal transmission wires of the patch electrode are hidden and fixed in the groove between the plastic plate and the skin-facing side of the plastic plate;

[0048] At the connection between the plastic plates, the signal transmission wires of the patch electrode are covered with a flexible plastic protective sleeve.

[0049] A flexible, retractable ring-shaped wristband is formed by connecting plastic plates with a highly elastic band. The wristband allows for adjustment of the overall circumference of the watch through the elasticity of the band.

[0050] Optionally, S2 includes:

[0051] S21: Establish an image coordinate system based on the captured photo, where the coordinates of the user's elbow joint in the captured photo are (picx, picy);

[0052] Convert the user's elbow coordinates to camera coordinates (camx, camy, camz):

[0053]

[0054] Where foc represents the camera's focal length, [ ] -1 Represents the inverse matrix;

[0055] Convert camera coordinates to world coordinates (worlx, worly, worz):

[0056]

[0057] Where [Rota|Tran] represents the extrinsic parameters of the camera, which are determined by the rotation matrix. Translation vectors [tranx trany tranz] T composition;

[0058] S22: Convert world coordinates (worlx, worly, worz) to coordinates in the dial coordinate system (dix, diy, diz);

[0059] S23: Calculate the user's arm length (darm) based on the coordinates in the dial coordinate system: darm = (dix) 2 +diy 2 +diz 2 ) 1 / 2 .

[0060] Optionally, S22 includes:

[0061] Based on the coordinates (wox, woy, woz) of the origin of the dial coordinate system in the world coordinate system, we obtain the vector WO from the origin of the world coordinate system to the origin of the dial coordinate system: WO = (wox, woy, woz).

[0062] In the dial coordinate system, the unit vector of the x-axis in the world coordinate system is represented as WDx = (wdialxx, wdialxy, wdialxz), the unit vector of the y-axis in the world coordinate system is represented as WDy = (wdialyx, wdialyy, wdialyz), and the unit vector of the z-axis in the world coordinate system is represented as WDz = (wdialzx, wdialzy, wdialzz).

[0063] Convert the world coordinates (worlx, worly, worlz) to the coordinates (dix, diy, diz) in the dial coordinate system;

[0064]

[0065] Optionally, S3 includes:

[0066] A second-order Butterworth bandpass filter was used to filter the electromyographic signal, preserving the main frequency components of the electromyographic signal;

[0067] A first-order high-pass filter was used to filter the electromyographic signal to remove DC offset and low-frequency drift; the filtered electromyographic signal was obtained.

[0068] This invention also provides an application system for an electromyography-based intelligent early warning watch for driver fatigue, comprising:

[0069] Inertial measurement module: Monitors user posture and starts working mode when the user is in a seated position; measures the real-time acceleration of the user's wrist movement and calculates the movement speed and displacement of the user's wrist; measures the rotational angular velocity of the user's wrist and, combined with quaternion algorithm, obtains the rotation angle of the user's wrist.

[0070] Signal acquisition module: Acquires electromyographic signals using patch electrodes;

[0071] Camera module: Responsible for taking pictures of the user's arm, locating the user's elbow joint based on the captured photos, and calculating the length of the user's arm;

[0072] Signal processing module: Filters the electromyographic signals acquired by the signal acquisition module;

[0073] Main control module: Extracts features from the filtered electromyography (EMG) signal and calculates the angular velocity of the user's elbow joint. When the sequential convolutional network determines that the filtered EMG signal indicates a fatigue state and the calculated angular velocity of the user's elbow joint exceeds a preset threshold, an early warning mechanism is triggered.

[0074] Power module: supplies power to the watch and controls the output power depending on whether the watch is in working mode.

[0075] Beneficial effects:

[0076] This invention overcomes the bottleneck of traditional fatigue monitoring technology based on electromyography (EMG) signals by using an innovative fusion judgment scheme of EMG characteristics and elbow joint range of motion, achieving accurate judgment and timely warning of driver fatigue. In this invention, the watch collects EMG signals from the user's wrist and elbow joint angular velocity in real time. EMG signals directly reflect the degree of muscle fatigue—when a driver enters a fatigued state, the arm muscles will exhibit tremors and decreased contraction strength, and the corresponding EMG signals will show characteristics such as reduced amplitude and reduced high-frequency components. Based on elbow joint angular velocity, sudden movements during driving can be captured. For example, when a driver's attention is distracted due to fatigue, "overcorrection" may occur during steering. Normally, when turning the steering wheel, the elbow joint completes the steering with a smooth angular velocity. However, in a fatigued state, the driver may experience a delayed judgment of the steering amplitude, initially resulting in understeering due to slow reaction, and then suddenly increasing the steering force. At this time, the elbow joint angular velocity will show a fluctuation of "slow at first and then suddenly fast." The fatigue assessment method based on electromyography characteristics and elbow joint range of motion utilizes the user's dual "physiological-behavioral" indicators, which can more comprehensively reflect the fatigue process than a single electrocardiogram signal. Attached Figure Description

[0077] Figure 1 This is a flowchart illustrating an application method of an electromyography-based intelligent early warning watch for fatigued driving, provided as an embodiment of the present invention. Detailed Implementation

[0078] The present invention will be further described below with reference to the accompanying drawings, but this is not intended to limit the present invention in any way. Any modifications or substitutions made based on the teachings of the present invention shall fall within the protection scope of the present invention.

[0079] Example 1:

[0080] An application method for an electromyography-based intelligent early warning watch for driver fatigue, such as... Figure 1 As shown, it includes the following steps:

[0081] The watch consists of a signal acquisition module, a camera module, an inertial measurement module, a signal processing module, a main control module, and a power supply module.

[0082] S1: The inertial measurement module monitors the user's posture. When the user is seated, it activates the working mode to measure the real-time acceleration of the user's wrist and calculate the wrist's velocity and displacement; it also measures the angular velocity of the user's wrist rotation and, using a quaternion algorithm, calculates the wrist rotation angle; the signal acquisition module then begins collecting electromyographic signals.

[0083] S11: The user wears the watch on their wrist, with the dial located at the center of the radiocarpal joint;

[0084] A dial coordinate system is established with the center of the radiocarpal joint as the origin. The plane passing through the center of the radiocarpal joint and parallel to the palm is defined as the xoz plane; the straight line perpendicular to the xoz plane and passing through the center of the radiocarpal joint is defined as the y-axis, with the positive direction of the y-axis pointing from the palm to the back of the hand; the direction parallel to the xoz plane, passing through the center of the radiocarpal joint, and from the inner side of the wrist (near the little finger) to the outer side (near the thumb) is defined as the x-axis; and the direction parallel to the xoz plane, passing through the center of the radiocarpal joint, and pointing from the elbow to the fingers is defined as the z-axis.

[0085] S12: The inertial measurement module measures the real-time acceleration of the user's wrist and calculates the speed and displacement of the user's wrist.

[0086] The inertial measurement module measures the angular velocity of the user's wrist rotation, and combines this with a quaternion algorithm to obtain the rotation angle of the user's wrist:

[0087] The velocity vector v(t) of the user's wrist at time t is expressed as:

[0088]

[0089] Where, v(t)=[v x (t),v y (t),v z (t)],v x (t),v y (t),v z (t) represents the user's wrist movement velocity in the x, y, and z directions at time t, respectively, and acce(τ) represents the user's wrist movement acceleration vector at time τ;

[0090] acce(τ)=[a x (τ),a y (τ),a z (τ)-pro(τ)·g]

[0091] Among them, a x (τ),a y (τ),a z (τ) represents the user's wrist acceleration in the x, y, and z directions at time τ, respectively, measured by the inertial measurement module; pro(τ) represents the gravitational acceleration to a. z The projection in the (τ) direction, where g represents the gravitational acceleration; at time 0, the user holds their arm vertically and initializes the watch, at which point pro(0) = 1;

[0092] The displacement vector dis(t) of the user's wrist at time t is expressed as:

[0093]

[0094] Where, dis(t) = [d x (t),d y (t),d z (t)],d x (t),d y (t),d z (t) represent the displacement of the user's wrist in the x, y, and z directions at time t, respectively;

[0095] When t = T, where T represents the time period, the absolute value of the user's wrist displacement, abdis, is represented as:

[0096] abdis=((d x (T)) 2 +(d y (T)) 2 +(d z (T)) 2 ) 1 / 2

[0097] Where, d x (T), d y (T), d z (T) represent the displacement of the user's wrist in the x, y, and z directions at time T, respectively;

[0098] Furthermore, at the initial moment, based on the initial attitude, the initial attitude quaternion q(0) is determined: q(0) = [1,0,0,0] T ,in,[] T Represents the transpose matrix;

[0099] The angular velocity vector ω(t) of the user's wrist at time t is expressed as:

[0100] ω(t)=[ω x (t),ω y (t),ω z (t)]

[0101] Where, ω x (t),ω y (t),ω z (t) represent the angular velocities of the user's wrist in the x, y, and z directions at time t, respectively;

[0102] The attitude quaternion at time t is q(t) = [q ω (t),q x (t),q y (t),q z (t)] T , satisfying (q ω (t)) 2 +(q x (t)) 2 +(q y (t)) 2 +(q z (t)) 2 =1, where q ω (t) represents the real part of the attitude quaternion at time t, q x (t) represents the component of the imaginary part x of the attitude quaternion at time t, q y (t) represents the component of the imaginary part y of the attitude quaternion at time t, q z (t) represents the component of the imaginary part z of the attitude quaternion at time t, specifically:

[0103]

[0104] allang(t)=((ω x (t-1)+ω x (t)·Δt) 2 +(ω y (t-1)+ω y (t)·Δt) 2 +(ω z (t-1)+ω z (t)·Δt) 2 ) 1 / 2

[0105] Where allang(t) represents the rotation angle of the user's wrist at time t, ω x (t-1),ω y (t-1),ω z (t-1) represents the angular velocity of the user's wrist in the x, y, and z directions at time t-1, respectively, Δt represents the time sampling interval, sin is the sine function, and cos is the cosine function;

[0106] Instantaneous rate of change of attitude quaternion at time t Represented as:

[0107]

[0108] in, The quaternion ω represents the attitude quaternion q(t) at time t and the angular velocity at time t. q The instantaneous rate of change under the action of (t), This represents quaternion multiplication, where q(t+1) represents the attitude quaternion at time t+1, and ω... q (t) represents the quaternion representation of the angular velocity at time t, ω q (t)=[0,ω x (t),ωy(t),ω z (t)] T ;

[0109] When t = T, the angle of rotation of the user's wrist is expressed as:

[0110]

[0111] Where θ represents the pitch angle of the user's wrist rotation. ψ represents the roll angle of the user's wrist rotation, y represents the yaw angle of the user's wrist rotation, and q represents the roll angle of the user's wrist rotation. ω (T) represents the real part of the attitude quaternion at time T, q x (T) represents the component of the imaginary part x of the attitude quaternion at time T, q y (T) represents the component of the imaginary part y of the attitude quaternion at time T, q z (T) represents the component of the imaginary part z of the attitude quaternion at time T;

[0112] The absolute value of the user's wrist rotation angle, abangle, is represented as:

[0113]

[0114] In this embodiment of the invention, after simplification, the attitude quaternion q(t+1) at time t+1 is expressed as:

[0115]

[0116] S13: When the absolute value of the user's wrist movement displacement is less than the activity threshold and the absolute value of the user's wrist rotation angle is less than the standing threshold, the user is determined to be in a sitting position.

[0117] Optionally, the signal acquisition module uses patch electrodes to acquire electromyographic signals:

[0118] The signal acquisition module includes patch electrodes and an adjustable wristband assembly;

[0119] The adjustable wristband assembly consists of a high-elasticity elastic band connecting multiple rigid or semi-rigid plastic plates; the plastic plates are in contact with the skin when worn, and patch electrodes are fixedly installed on their inner surfaces.

[0120] The signal transmission wires of the patch electrode are hidden and fixed inside the plastic plate on which it is located (in the groove between the skin side and the plastic plate body);

[0121] At the connection between the plastic plates (i.e., the joint between the elastic band and the plastic plate), the signal transmission wires of the patch electrode are covered with a flexible plastic protective sleeve to provide stress relief and mechanical protection, and to prevent the wires from being bent and damaged.

[0122] A flexible, retractable ring-shaped wristband is formed by connecting plastic plates with a highly elastic band. The wristband allows for adjustment of the overall circumference of the watch through the elasticity of the band.

[0123] S2: The camera module is responsible for taking pictures of the user's arm, locating the user's elbow joint based on the pictures, and calculating the length of the user's arm.

[0124] S21: Establish an image coordinate system based on the captured photo, where the coordinates of the user's elbow joint in the captured photo are (picx, picy);

[0125] Convert the coordinates of the user's elbow joint in the captured photo to camera coordinates (camx, camy, camz):

[0126]

[0127] Where foc represents the camera's focal length, [ ] -1 Represents the inverse matrix;

[0128] Convert camera coordinates to world coordinates (worlx, worly, worz):

[0129]

[0130] Where [Rota|Tran] represents the extrinsic parameters of the camera, which are determined by the rotation matrix. Translation vectors [tranx trany tranz] T composition;

[0131] S22: Convert world coordinates (worlx, worly, worlz) to dial coordinates (dix, diy, diz):

[0132] Based on the coordinates (wox, woy, woz) of the origin of the dial coordinate system in the world coordinate system, we obtain the vector WO from the origin of the world coordinate system to the origin of the dial coordinate system: WO = (wox, woy, woz).

[0133] In the dial coordinate system, the unit vector of the x-axis in the world coordinate system is represented as WDx = (wdialxx, wdialxy, wdialxz), the unit vector of the y-axis in the world coordinate system is represented as WDy = (wdialyx, wdialyy, wdialyz), and the unit vector of the z-axis in the world coordinate system is represented as WDz = (wdialzx, wdialzy, wdialzz).

[0134] Convert the world coordinates (worlx, worly, worlz) to the coordinates (dix, diy, diz) in the dial coordinate system;

[0135]

[0136] S23: Calculate the length of the user's arm (darm) based on the coordinates in the dial coordinate system: darm = (dix) 2 +diy 2 +diz 2 ) 1 / 2 .

[0137] S3: The signal processing module filters the acquired electromyographic signals to obtain filtered electromyographic signals;

[0138] A second-order Butterworth bandpass filter was used to filter the electromyographic signal, preserving the main frequency components of the electromyographic signal;

[0139] A first-order high-pass filter was used to filter the electromyographic signal to remove DC offset and low-frequency drift.

[0140] In this embodiment of the invention, the transfer function H of the second-order Butterworth bandpass filter bt (fre) is represented as:

[0141]

[0142] Where fre represents the signal frequency, ω center The center angular frequency is represented by Qua, the quality factor is represented by ω. low ω represents the low-frequency cutoff angular frequency. high This indicates the high-frequency cutoff angular frequency.

[0143] S4: Feature extraction is performed on the filtered electromyography (EMG) signal using a temporal convolutional network; the main control module calculates the user's elbow joint angular velocity based on the user's wrist movement speed collected by the inertial measurement module and the user's arm length obtained by the camera module; the main control module triggers an early warning mechanism when the following two conditions are met simultaneously: first, the analysis result of the temporal convolutional network on the filtered EMG signal indicates a fatigue state; second, the calculated user's elbow joint angular velocity is greater than a preset threshold.

[0144] In this embodiment of the invention, the main control module includes a signal processing section and a display screen, with the display screen located on the front of the dial;

[0145] The main control module extracts features from the filtered electromyography (EMG) signals using a temporal convolutional network. Simultaneously, based on the user's wrist movement speed acquired by the inertial measurement module and the user's arm length obtained by the camera module, the main control module calculates the angular velocity of the user's elbow joint. The main control module triggers an early warning mechanism when two conditions are met: first, the temporal convolutional network determines the filtered EMG signal to be in a state of fatigue; second, the calculated angular velocity of the user's elbow joint exceeds a preset threshold.

[0146] It should be noted that, in this embodiment of the invention, the temporal convolutional network is a deep learning model suitable for time series processing, which adopts an architecture design that combines causal convolution and dilated convolution; wherein, causal convolution ensures that the output at the current time depends only on the previous current time and earlier elements; dilated convolution constructs a larger effective filter by introducing zero padding or skipping part of the input in the original filter;

[0147] In this embodiment of the invention, the filtered electromyographic signal sequence is first divided into signal segments of equal length to form an electromyographic signal matrix Filsig:

[0148]

[0149] Among them, fil_sig 1,1 ,fil_sig 1,2 ,…,fil_sig 1,snum This represents the equal-length signal segment obtained after dividing the filtered electromyography signal sequence with sequence number 1. 2,1 ,fil_sig 2,2 ,…,fil_sig 2,snum This represents the equal-length signal segments obtained after dividing the filtered electromyography (EMG) signal sequence with sequence number 2. snum represents the number of signal segments, and tnum represents the number of filtered EMG signal sequences.

[0150] The output of the causal convolution layer 1 (Cause)lay The output of the dilated convolution in the 1st layer (Expan) lay They are represented as follows:

[0151] Cause lay =weight c ·Expan lay-1 +bias c

[0152] Expan lay =weight e ·Cause lay-1 ′+bias e

[0153] Among them, Expan lay-1 This represents the output of the (-1)th layer dilated convolution, weight c The weights represent the weights of the causal convolution kernel. e The bias represents the weights of the dilated convolution kernel. c The bias represents the causal convolution kernel. e This represents the bias of the expanded convolution kernel; and satisfies Expan0 = Filsig; Cause lay-1 This represents the output of the (lay-1)th causal convolution layer, Cause lay-1 ' is in Cause lay-1 Based on this, an expansion factor is introduced to expand it to obtain the result;

[0154] The angular velocity of the user's elbow joint, elbow, is represented as:

[0155] elbow=((v x (t) / darm) 2 +(v y (t) / darm) 2 ) 1 / 2 .

[0156] In this embodiment of the invention, the power module is responsible for supplying power to the watch and controlling the output power according to whether the watch is in working mode.

[0157] Example 2: The present invention also provides an application system for an electromyography-based intelligent early warning watch for driver fatigue, comprising the following six modules:

[0158] Inertial measurement module: Monitors user posture and starts working mode when the user is in a seated position; measures the real-time acceleration of the user's wrist movement and calculates the movement speed and displacement of the user's wrist; measures the rotational angular velocity of the user's wrist and, combined with quaternion algorithm, obtains the rotation angle of the user's wrist.

[0159] Signal acquisition module: Acquires electromyographic signals using patch electrodes;

[0160] Camera module: Responsible for taking pictures of the user's arm, locating the user's elbow joint based on the captured photos, and calculating the length of the user's arm;

[0161] Signal processing module: Filters the electromyographic signals acquired by the signal acquisition module;

[0162] Main control module: Extracts features from the filtered electromyography (EMG) signal and calculates the angular velocity of the user's elbow joint. When the sequential convolutional network determines that the filtered EMG signal indicates a fatigue state and the calculated angular velocity of the user's elbow joint exceeds a preset threshold, an early warning mechanism is triggered.

[0163] Power module: supplies power to the watch and controls the output power depending on whether the watch is in working mode.

[0164] It should be noted that the sequence numbers of the above embodiments of the present invention are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0165] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0166] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. An application method of a myoelectric intelligent early warning watch for fatigue driving, characterized in that, The method comprises: S1: the inertial measurement module is responsible for monitoring the user posture, and when the user is in a sitting posture, a working mode is started, the real-time moving acceleration of the user's wrist is measured, the moving speed and displacement of the user's wrist are calculated, the rotational angular velocity of the user's wrist is measured, and the rotational angle of the user's wrist is obtained by combining a quaternion algorithm; after the working mode is started, the signal acquisition module starts to collect the electromyographic signal by using the patch electrode; S2: the camera module is responsible for shooting the user's arm, and the user's elbow joint is positioned based on the shot photo, and the length of the user's arm is obtained by calculation; S3: the signal processing module filters the collected electromyographic signal to obtain the filtered electromyographic signal; S4: the filtered electromyographic signal is extracted by a time sequence convolution network; the main control module calculates the activity angular velocity of the user's elbow joint based on the moving speed of the user's wrist collected by the inertial measurement module and the length of the user's arm obtained by the camera module; when the following two conditions are met at the same time, the main control module triggers the early warning mechanism: one is that the analysis result of the filtered electromyographic signal by the time sequence convolution network is determined as a fatigue state; and two is that the calculated activity angular velocity of the user's elbow joint is greater than a preset threshold value; The step S1 comprises: S11: the user wears the watch on the wrist, wherein the watch dial is located at the center of the radiocarpal joint; Establish a dial coordinate system with the center of the radiocarpal joint as the origin. The plane passing through the center of the radiocarpal joint and parallel to the palm is considered as... Plane; perpendicular to A plane, a straight line passing through the center of the radiocarpal joint as The axis, from the palm to the back of the hand, is Positive axis; parallel to The plane, passing through the center of the radiocarpal joint, is set in the direction from the little finger side of the wrist to the thumb side. Axial direction; parallel to The plane, passing through the center of the radiocarpal joint, and the direction from the elbow towards the fingers is set as... Axial direction; S12: the inertial measurement module measures the real-time moving acceleration of the user's wrist, calculates the moving speed and displacement of the user's wrist, and obtains the rotational angle of the user's wrist by combining a quaternion algorithm; S13: when the absolute value of the moving displacement of the user's wrist is less than an activity threshold value, and the absolute value of the rotational angle of the user's wrist is less than a standing threshold value, it is determined that the user is in a sitting posture.

2. The application method of the fatigue driving-oriented myoelectric intelligent early warning watch according to claim 1, characterized in that, The step S12 comprises: the speed vector of the movement of the user's wrist at the moment is represented as: wherein, , respectively represent the moving speed of the user's wrist in the direction of the time instant, represent the moving acceleration vector of the user's wrist at the time instant;​ wherein, respectively represent the movement acceleration of the user's wrist in the direction of the watch face at time t, measured by an inertial measurement module; represents the projection of the gravitational acceleration onto the direction of the watch face, represents the gravitational acceleration; at time t = 0, the user holds the arm vertically and initializes the watch, at which time ;​ displacement vector of the user's wrist at the time instant is represented as: wherein, , respectively represent the movement displacement of the user's wrist in the direction of the time. When Time, represents the absolute value of the movement displacement of the user's wrist is represented as: wherein, , , respectively represent the movement displacement of the user's wrist in the direction at the moment At an initial time instant, an initial pose quaternion is determined from the initial pose : wherein denotes a transposed matrix; the rotational angular velocity vector of the user's wrist at the moment is represented as: wherein respectively represent the wrist of the user at the rotational angular velocity in the direction the time instance attitude quaternion , satisfies wherein, denotes the real part of the time instance attitude quaternion, denotes the component of the imaginary part of the time instance attitude quaternion , denotes the component of the imaginary part of the time instance attitude quaternion , denotes the component of the imaginary part of the time instance attitude quaternion , in particular: wherein, denotes the rotational angle of the user's wrist at the time instant, denotes the rotational angular velocity of the user's wrist in the direction at the time instant, denotes denotes the time sampling interval, sin is the sine function, and cos is the cosine function; Instantaneous rate of change of the time-varying attitude quaternion is represented as: wherein represents instantaneous attitude quaternion at instantaneous angular velocity quaternion under the influence of represents quaternion multiplication, represents instantaneous attitude quaternion, represents instantaneous angular velocity quaternion representation, ; When the rotation angle of the user's wrist is represented as: wherein represents a pitch angle of the user's wrist rotation, represents a roll angle of the user's wrist rotation, represents a yaw angle of the user's wrist rotation, represents a real component of the time instant attitude quaternion, represents an imaginary component of the time instant attitude quaternion represents a real component of the time instant attitude quaternion, represents an imaginary component of the time instant attitude quaternion represents an imaginary component of the time instant attitude quaternion represents an imaginary component of the time instant attitude quaternion. The absolute value of the rotation angle of the user's wrist is represented as: 。 3.The application method of the fatigue driving-oriented myoelectric intelligent warning watch according to claim 1, characterized in that, The step S1 further comprises: The signal acquisition module comprises a patch electrode and an adjustable wristband assembly; The adjustable wristband assembly is composed of a plurality of plastic plates connected by a high-elasticity elastic band, the plastic plates contact the skin when worn, and the inner side surface of the plastic plates is fixedly installed with the patch electrode; wherein the plastic plates are rigid or semi-rigid plastic plates; The signal transmission lead of the patch electrode is hidden and fixed in the groove between the plastic plate where the patch electrode is located and the skin side of the plastic plate; At the connection between the plastic plates, the signal transmission lead of the patch electrode is externally covered with a flexible plastic protective sleeve; The high-elasticity elastic band connects the plastic plates to form a flexible and retractable annular wristband, and the wristband adjusts the overall wearing circumference of the watch through the elasticity of the elastic band.

4. The application method of the fatigue driving-oriented myoelectric intelligent early warning watch according to claim 1, characterized in that, The step S2 comprises: S21: establish an image coordinate system based on the photographed photo, and the coordinate of the elbow joint of the user in the photographed photo is ; Converting coordinates of a user's elbow joint to camera coordinates : wherein denotes the camera focal length, denotes the inverse matrix; Converting camera coordinates to world coordinates : wherein, represents the extrinsic parameter of the camera, which is composed of a rotation matrix and a translation vector ; S22: convert the world coordinates into coordinates in the watch face coordinate system S22: convert the world coordinates into coordinates in the watch face coordinate system ; S23: Calculate the length of the user's arm based on the coordinates in the watch face coordinate system : .

5. The application method of the fatigue driving-oriented myoelectric intelligent early warning watch according to claim 4, characterized in that, The step S22 comprises: based on the coordinates of the origin of the dial coordinate system in the world coordinate system , obtaining a vector from the origin of the world coordinate system to the origin of the dial coordinate system : ; The unit vector of the axis in the world coordinate system is represented as The unit vector of the axis in the world coordinate system is represented as , The unit vector of the axis in the world coordinate system is represented as , The unit vector of the axis in the world coordinate system is represented as ; converts the world coordinates into coordinates in the coordinate system of the dial ; 。 6. The application method of the fatigue driving-oriented myoelectric intelligent early warning watch according to claim 1, characterized in that, The step S3 comprises: The electromyographic signal is filtered by using a second-order Butterworth band-pass filter to retain the main frequency components of the electromyographic signal; The electromyographic signal is filtered by using a first-order high-pass filter to remove the direct current offset and low-frequency drift in the electromyographic signal; The filtered electromyographic signal is obtained.

7. A system for application of an electromyographic intelligent early warning watch for fatigue driving, characterized in that, It comprises: The inertial measurement module monitors the user posture, and when the user is in a sitting posture, a working mode is started; The real-time moving acceleration of the user's wrist is measured, the moving speed and displacement of the user's wrist are calculated, the rotational angular velocity of the user's wrist is measured, and the rotational angle of the user's wrist is obtained by combining a quaternion algorithm; Signal acquisition module: use patch electrode to collect electromyography signal; Camera module: responsible for shooting user's arm, positioning user's elbow joint based on shooting photo, calculating user's arm length; Signal processing module: filter electromyography signal collected by signal acquisition module; Main control module: extract feature of filtered electromyography signal, calculate activity angular velocity of user's elbow joint; When time sequence convolution network determines that fatigue state based on analysis result of filtered electromyography signal, and activity angular velocity of user's elbow joint calculated is greater than preset threshold, trigger early warning mechanism; Power module: power supply for watch, control output power according to whether watch is working mode; To realize the electromyography intelligent early warning watch for fatigue driving as claimed in any one of claims 1-6.

Images