Application method and system of myoelectricity intelligent early warning watch for fatigue driving

Abstract

The invention discloses an application method and system of a myoelectricity intelligent early warning watch for fatigue driving, and the method comprises the steps: monitoring the posture of a user through an inertia measurement module, starting a working mode when the user is in a sitting posture state, and obtaining the rotation angle of the wrist of the user; the signal acquisition module acquires an electromyographic signal by using a patch electrode; the camera module obtains the arm length of the user through calculation; the signal processing module filters the electromyographic signals; and the main control module performs feature extraction on the electromyographic signals and calculates the motion angular velocity of the elbow joint of the user. According to the method, the user state is accurately judged based on the time sequence convolutional network and the angular velocity of the elbow joint movement of the user; when the user is tired, the watch gives an early warning to remind the user, and dangerous events are avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of myoelectric watches, and in particular to an application method and system of a myoelectric intelligent warning watch for fatigue driving. Background Art

[0002] Determining driver fatigue is crucial to road safety and a key step in preventing accidents caused by fatigued driving. Statistics show that fatigued driving accounts for 15%-20% of all traffic accidents worldwide, and these accidents are often more severe. From a driving behavioral perspective, fatigued drivers experience distraction, decreased reaction speed, weakened judgment, and even short sleep, significantly increasing the risk of accidents. Therefore, designing wearable devices that can detect user fatigue and provide early warnings of dangerous situations is crucial.

[0003] Currently, some smart bracelets on the market use sensors to monitor fatigue. For example, one smart bracelet is equipped with 16 high-sensitivity sensors, similar to those used in lie detectors, that can accurately detect the wearer's heart rate and skin conductivity. By analyzing this data, it can determine whether the driver is fatigued. A certain car company's smartwatch also has a fatigue monitoring function. It can monitor heart rate, blood oxygen, fatigue, and sleep 24 / 7. If the monitored data reaches a high-risk level, the watch will issue an alert.

[0004] Most of these wearable devices use ECG signal characteristics to determine fatigue status. ECG signals are easily distorted by multiple factors. For example, while driving, driver manipulation of the steering wheel, shifting gears, and other factors can interfere with muscle electrical signals, leading to deviations in key indicators such as heart rate variability. In high-temperature environments, perspiration reduces electrode conductivity; at high altitudes, hypoxia can lead to a compensatory increase in heart rate. Furthermore, improperly worn device positioning or a loose strap can directly affect signal acquisition quality. Summary of the Invention

[0005] In order to solve the problem of insufficient accuracy in fatigue status judgment based on electrocardiogram signal characteristics, the present invention provides an application method and system for an early warning watch that simultaneously utilizes electromyography characteristics and elbow joint mobility to judge fatigue status, thereby effectively improving the accuracy of early warnings and avoiding the occurrence of dangerous events.

[0006] The technical solution proposed in the present invention is an application method of a myoelectric intelligent warning watch for fatigue driving, comprising the following steps:

[0007] S1: The inertial measurement module is responsible for monitoring the user's posture. When the user is in a sitting position, the working mode is activated to measure the real-time acceleration of the user's wrist and calculate the movement speed and displacement of the user's wrist. The angular velocity of the user's wrist is measured and combined with the quaternion algorithm to obtain the rotation angle of the user's wrist. The signal acquisition module begins to collect electromyographic signals using patch electrodes.

[0008] S2: Uses the camera module to capture the user's arm, locates the user's elbow joint based on the captured photo, and calculates the length of the user's arm;

[0009] S3: The signal processing module filters the collected electromyographic signal to obtain a filtered electromyographic signal;

[0010] S4: The filtered electromyographic signal is subjected to feature extraction through a time series convolutional network. The main control module calculates the angular velocity of the user's elbow joint based on the wrist movement speed collected by the inertial measurement module and the arm length obtained by the camera module. The main control module triggers the early warning mechanism when the following two conditions are met at the same time: first, the analysis result of the filtered electromyographic signal by the time series convolutional network is judged to be a fatigue state; second, the calculated angular velocity of the user's elbow joint is greater than a preset threshold.

[0011] Optionally, the S1 includes:

[0012] S11: The user wears the watch on the wrist, with the dial located at the center of the radiocarpal joint;

[0013] With the center of the radiocarpal joint as the coordinate origin, establish a dial coordinate system. The plane passing through the center of the radiocarpal joint and parallel to the palm is the xoz plane. The line perpendicular to the xoz plane and passing through the center of the radiocarpal joint is the y-axis, with the direction from the palm to the back of the hand as the positive y-axis direction. The direction parallel to the xoz plane, passing through the center of the radiocarpal joint, and from the wrist near the little finger to the wrist near the thumb is set as the x-axis direction. The direction parallel to the xoz plane, passing through the center of the radiocarpal joint, and from the elbow to the fingers is set as the z-axis direction.

[0014] S12: The inertial measurement module measures the real-time acceleration of the user's wrist and calculates the speed and displacement of the user's wrist. Combined with the quaternion algorithm, the rotation angle of the user's wrist is obtained.

[0015] S13: When the absolute value of the movement displacement of the user's wrist is less than the activity threshold, and the absolute value of the rotation angle of the user's wrist is less than the standing threshold, it is determined that the user is in a sitting state.

[0016] Optionally, the S12 includes:

[0017] The movement velocity vector v(t) of the user's wrist at time t is expressed as:

[0018]

[0019] Where v(t)=[v x (t),v y (t),v z (t)],v x (t),v y (t),v z (t) represents the movement speed of the user's wrist in the x, y, and z directions at time t, respectively, and acce(τ) represents the movement acceleration vector of the user's wrist at time τ;

[0020] acce(τ)=[a x (τ),a y (τ),a z (τ)-pro(τ)·g]

[0021] Among them, a x (τ),a y (τ),a z (τ) represents the acceleration of the user's wrist in the x, y, and z directions at time τ, measured by the inertial measurement module; pro(τ) represents the acceleration due to gravity to a z (τ) direction projection, g represents the acceleration of gravity; at time 0, the user puts his arm vertically to initialize the watch, at this time pro(0) = 1;

[0022] The displacement vector dis(t) of the user's wrist at time t is expressed as:

[0023]

[0024] Where dis(t)=[d x (t),d y (t),d z (t)],d x (t),d y (t),d z (t) represents the displacement of the user's wrist in the x, y, and z directions at time t, respectively;

[0025] When t=T, T represents the time period, and the absolute value of the user's wrist displacement abdis is expressed as:

[0026] abdis=((d x (T)) 2 +(d y (T)) 2 +(d z (T)) 2 ) 1 / 2

[0027] Among them, d x (T), d y (T), d z (T) represents the displacement of the user's wrist in the x, y, and z directions at time T, respectively;

[0028] At the initial moment, according to the initial posture, determine the initial posture quaternion q(0): q(0) = [1,0,0,0] T ,in[] T represents the transposed matrix;

[0029] The angular velocity vector ω(t) of the user's wrist at time t is expressed as:

[0030] ω(t)=[ω x (t),ω y (t),ω z (t)]

[0031] Among them, ω x (t),ω y (t),ω z (t) represents the angular velocity of the user's wrist in the x, y, and z directions at time t, respectively;

[0032] Attitude quaternion q(t) = pq at time t ω (t),q x (t),q y (t),q z (t)] T , satisfying (q ω (t)) 2 +(q x (t)) 2 +(q y (t)) 2 +(q z (t)) 2 =1, where q ω (t) represents the real part of the attitude quaternion at time t, q x (t) represents the component of the imaginary part x of the attitude quaternion at time t, q y (t) represents the component of the imaginary part y of the attitude quaternion at time t, q z (t) represents the component of the imaginary part z of the attitude quaternion at time t, specifically:

[0033]

[0034] allang(t)=((ω x (t-1)+ω x (t)·Δt) 2 +(ω y (t-1)+ωy (t)·Δt) 2 +(ω z (t-1)+ω z (t)·Δt) 2 ) 1 / 2

[0035] Among them, allang(t) represents the rotation angle of the user's wrist at time t, ω x (t-1),ω y (t-1),ω z (t-1) represents the angular velocity of the user's wrist in the x, y, and z directions at time t-1, Δt represents the time sampling interval, sin represents the sine function, and cos represents the cosine function;

[0036] The instantaneous rate of change of the attitude quaternion at time t Expressed as:

[0037]

[0038] in, The quaternion ω representing the angular velocity of the attitude quaternion q(t) at time t q (t) The instantaneous rate of change under the action of represents quaternion multiplication, q(t+1) represents the attitude quaternion at time t+1, ω q (t) represents the quaternion representation of the angular velocity at time t, ω q (t)=[0,ω x (t),ω y (t),ω z (t)] T ;

[0039] When t=T, the rotation angle of the user's wrist is expressed as:

[0040]

[0041] Where θ represents the pitch angle of the user's wrist rotation, represents the roll angle of the user's wrist rotation, ψ represents the yaw angle of the user's wrist rotation, and q ω (T) represents the real part of the attitude quaternion at time T, q x (T) represents the component of the imaginary part x of the attitude quaternion at time T, q y (T) represents the component of the imaginary part y of the attitude quaternion at time T, q z (T) represents the component of the imaginary part z of the attitude quaternion at time T;

[0042] The absolute value of the user's wrist rotation angle abangle is expressed as:

[0043]

[0044] Optionally, the S1 includes:

[0045] The signal acquisition module includes patch electrodes and an adjustable wristband assembly;

[0046] The adjustable wristband assembly is composed of a plurality of plastic plates connected by a high-elasticity elastic band. When worn, the plastic plates contact the skin, and patch electrodes are fixedly mounted on the inner surfaces of the plastic plates. The plastic plates are rigid or semi-rigid plastic plates.

[0047] The signal transmission wire of the patch electrode is hidden and fixed in the groove between the skin-facing side of the plastic plate and the plastic plate;

[0048] At the connection between the plastic plates, the signal transmission wires of the patch electrodes are covered with a flexible plastic protective cover;

[0049] A flexible and retractable ring-shaped wristband is formed by connecting the plastic plates with a high-elasticity elastic band. The elasticity of the elastic band enables the adjustment of the overall wearing circumference of the watch.

[0050] Optionally, the S2 includes:

[0051] S21: establishing an image coordinate system based on the photograph, where the coordinates of the user's elbow joint in the photograph are (picx, pici);

[0052] Convert the coordinates of the user's elbow joint to camera coordinates (camx, camy, camz):

[0053]

[0054] Among them, foc represents the focal length of the camera, [] -1 represents the inverse matrix;

[0055] Convert camera coordinates to world coordinates (worlx, worly, worlz):

[0056]

[0057] Among them, [Rota|Tran] represents the external parameters of the camera, which are the rotation matrix and translation vector [tranx trany tranz] T composition;

[0058] S22: Convert the world coordinates (worlx, worly, worlz) into the coordinates of the dial coordinate system (dix, diy, diz);

[0059] S23: Calculate the user's arm length darm based on the coordinates in the dial coordinate system: darm=(dix 2 +DIY 2 +diz 2 ) 1 / 2 .

[0060] Optionally, the S22 includes:

[0061] Based on the coordinates (wox, woy, woz) corresponding to the origin of the dial coordinate system in the world coordinate system, obtain the vector WO from the origin of the world coordinate system to the origin of the dial coordinate system: WO = (wox, woy, woz);

[0062] In the dial coordinate system, the unit vector of the x-axis in the world coordinate system is expressed as WDx = (wdialxx, wdialxy, wdialxz), the unit vector of the y-axis in the world coordinate system is expressed as WDy = (wdialyx, wdialyy, wdialyz), and the unit vector of the z-axis in the world coordinate system is expressed as WDz = (wdialzx, wdialzy, wdialzz);

[0063] Convert the world coordinates (worlx, worly, worlz) into the coordinates of the dial coordinate system (dix, diy, diz);

[0064]

[0065] Optionally, the S3 includes:

[0066] A second-order Butterworth bandpass filter is used to filter the EMG signal to retain the main frequency components of the EMG signal;

[0067] A first-order high-pass filter is used to filter the electromyographic signal to remove the DC offset and low-frequency drift in the electromyographic signal; and a filtered electromyographic signal is obtained.

[0068] The present invention also provides an application system for a myoelectric intelligent warning watch for fatigue driving, comprising:

[0069] Inertial measurement module: monitors the user's posture and activates the working mode when the user is in a sitting position; measures the real-time acceleration of the user's wrist and calculates the movement speed and displacement of the user's wrist; measures the angular velocity of the user's wrist and, combined with the quaternion algorithm, calculates the rotation angle of the user's wrist;

[0070] Signal acquisition module: uses patch electrodes to collect electromyographic signals;

[0071] Camera module: responsible for photographing the user's arm, locating the user's elbow joint based on the photographed image, and calculating the length of the user's arm;

[0072] Signal processing module: filters the electromyographic signals collected by the signal acquisition module;

[0073] Main control module: This module extracts features from the filtered EMG signals and calculates the angular velocity of the user's elbow joint. If the temporal convolutional network analyzes the filtered EMG signals and determines they are in a fatigue state, and the calculated angular velocity of the user's elbow joint is greater than a preset threshold, an early warning mechanism is triggered.

[0074] Power module: Provides power to the watch and controls the output power according to whether the watch is in working mode.

[0075] Beneficial effects:

[0076] This invention breaks through the bottleneck of traditional fatigue monitoring technology based on electrocardiogram signals through an innovative fusion judgment scheme that combines electromyographic characteristics with elbow joint range of motion, achieving accurate judgment and timely warning of the driver's fatigue state. In this invention, the watch collects electromyographic signals and elbow joint angular velocity at the user's wrist in real time. The electromyographic signals can directly reflect the degree of muscle fatigue. When the driver enters a fatigued state, the arm muscles will tremble and the contraction force will decrease. The corresponding electromyographic signals will show characteristics such as reduced amplitude and reduced high-frequency components. Based on the elbow joint angular velocity, sudden movements during driving operations can be captured. For example, when the driver's attention is distracted due to fatigue, "overcorrection" movements may occur during steering operations. That is, when turning the steering wheel during normal driving, the elbow joint will complete the steering at a steady angular velocity. However, in a fatigued state, the driver may delay the judgment of the steering amplitude, first understeering due to slow reaction, and then suddenly increase the steering force. At this time, the elbow joint angular velocity will show a "slow first, then fast" fluctuation. The fatigue state judgment method based on electromyographic characteristics and elbow joint mobility utilizes the user's "physiological-behavioral" dual indicators and can more comprehensively reflect the fatigue process than a single electrocardiogram signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 A flowchart of an application method of a myoelectric intelligent warning watch for fatigue driving provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0078] The present invention will be further described below with reference to the accompanying drawings, but the present invention is not limited in any way. Any changes or substitutions made based on the teachings of the present invention fall within the scope of protection of the present invention.

[0079] Example 1:

[0080] An application method of myoelectric intelligent warning watch for fatigue driving, such as Figure 1 As shown, the following steps are included:

[0081] The watch consists of a signal acquisition module, a camera module, an inertial measurement module, a signal processing module, a main control module, and a power module;

[0082] S1: The inertial measurement module is responsible for monitoring the user's posture. When the user is in a sitting position, the working mode is activated to measure the real-time acceleration of the user's wrist and calculate the movement speed and displacement of the user's wrist. The angular velocity of the user's wrist is measured and combined with the quaternion algorithm to obtain the rotation angle of the user's wrist. The signal acquisition module starts to collect electromyographic signals:

[0083] S11: The user wears the watch on the wrist, with the dial located at the center of the radiocarpal joint;

[0084] The dial coordinate system is established with the center of the radiocarpal joint as the coordinate origin. The plane passing through the center of the radiocarpal joint and parallel to the palm is the xoz plane. The straight line perpendicular to the xoz plane and passing through the center of the radiocarpal joint is the y-axis, and the direction from the palm to the back of the hand is the positive y-axis direction. The direction parallel to the xoz plane, passing through the center of the radiocarpal joint, and from the inside of the wrist (near the little finger) to the outside (near the thumb) is set as the x-axis direction. The direction parallel to the xoz plane, passing through the center of the radiocarpal joint, and from the elbow to the fingers is set as the z-axis direction.

[0085] S12: The inertial measurement module measures the real-time acceleration of the user's wrist and calculates the movement speed and displacement of the user's wrist;

[0086] The inertial measurement module measures the angular velocity of the user's wrist and, combined with the quaternion algorithm, obtains the rotation angle of the user's wrist:

[0087] The movement velocity vector v(t) of the user's wrist at time t is expressed as:

[0088]

[0089] Where v(t)=[v x (t),v y (t),v z (t)],v x (t),v y (t),v z (t) represents the movement speed of the user's wrist in the x, y, and z directions at time t, respectively, and acce(τ) represents the movement acceleration vector of the user's wrist at time τ;

[0090] acce(τ)=[a x (τ),a y (τ),a z (τ)-pro(τ)·g]

[0091] Among them, a x (τ),a y (τ),a z (τ) represents the acceleration of the user's wrist in the x, y, and z directions at time τ, measured by the inertial measurement module; pro(τ) represents the acceleration due to gravity to a z (τ) direction projection, g represents the acceleration of gravity; at time 0, the user puts his arm vertically to initialize the watch, at this time pro(0) = 1;

[0092] The displacement vector dis(t) of the user's wrist at time t is expressed as:

[0093]

[0094] Where dis(t)=[d x (t),d y (t),d z (t)],d x (t),d y (t),d z (t) represents the displacement of the user's wrist in the x, y, and z directions at time t, respectively;

[0095] When t=T, T represents the time period, and the absolute value of the user's wrist displacement abdis is expressed as:

[0096] abdis=((d x (T)) 2 +(d y (T)) 2 +(d z (T)) 2 ) 1 / 2

[0097] Among them, d x (T), d y (T), d z (T) represents the displacement of the user's wrist in the x, y, and z directions at time T, respectively;

[0098] Furthermore, at the initial moment, according to the initial posture, the initial posture quaternion q(0) is determined: q(0) = [1, 0, 0, 0] T ,in,[] T represents the transposed matrix;

[0099] The angular velocity vector ω(t) of the user's wrist at time t is expressed as:

[0100] ω(t)=[ω x (t),ω y (t),ω z (t)]

[0101] Among them, ω x (t),ω y (t),ω z (t) represents the angular velocity of the user's wrist in the x, y, and z directions at time t, respectively;

[0102] Attitude quaternion q(t) = [q ω (t),q x (t),q y (t),q z (t)] T , satisfying (q ω (t)) 2 +(q x (t)) 2 +(q y (t)) 2 +(q z (t)) 2 =1, where q ω (t) represents the real part of the attitude quaternion at time t, q x (t) represents the component of the imaginary part x of the attitude quaternion at time t, q y (t) represents the component of the imaginary part y of the attitude quaternion at time t, q z (t) represents the component of the imaginary part z of the attitude quaternion at time t, specifically:

[0103]

[0104] allang(t)=((ω x (t-1)+ω x (t)·Δt) 2 +(ω y (t-1)+ω y (t)·Δt) 2 +(ω z (t-1)+ω z (t)·Δt) 2 ) 1 / 2

[0105] Among them, allang(t) represents the rotation angle of the user's wrist at time t, ω x (t-1),ω y (t-1),ω z (t-1) represents the angular velocity of the user's wrist in the x, y, and z directions at time t-1, Δt represents the time sampling interval, sin represents the sine function, and cos represents the cosine function;

[0106] The instantaneous rate of change of the attitude quaternion at time t Expressed as:

[0107]

[0108] in, The quaternion ω representing the angular velocity of the attitude quaternion q(t) at time t q (t) The instantaneous rate of change under the action of represents quaternion multiplication, q(t+1) represents the attitude quaternion at time t+1, ω q (t) represents the quaternion representation of the angular velocity at time t, ω q (t)=[0,ω x (t),ωy(t),ω z (t)] T ;

[0109] When t=T, the rotation angle of the user's wrist is expressed as:

[0110]

[0111] Where θ represents the pitch angle of the user's wrist rotation, represents the roll angle of the user's wrist rotation, ψ represents the yaw angle of the user's wrist rotation, and q ω (T) represents the real part of the attitude quaternion at time T, q x (T) represents the component of the imaginary part x of the attitude quaternion at time T, q y (T) represents the component of the imaginary part y of the attitude quaternion at time T, q z (T) represents the component of the imaginary part z of the attitude quaternion at time T;

[0112] The absolute value of the user's wrist rotation angle abangle is expressed as:

[0113]

[0114] In the embodiment of the present invention, after simplification, the attitude quaternion q(t+1) at time t+1 is expressed as:

[0115]

[0116] S13: When the absolute value of the movement displacement of the user's wrist is less than the activity threshold, and the absolute value of the rotation angle of the user's wrist is less than the standing threshold, it is determined that the user is in a sitting state.

[0117] Optionally, the signal acquisition module uses patch electrodes to collect electromyographic signals:

[0118] The signal acquisition module includes patch electrodes and an adjustable wristband assembly;

[0119] The adjustable wristband assembly is composed of a plurality of rigid or semi-rigid plastic plates connected by a highly elastic elastic band; the plastic plates are in contact with the skin when worn, and patch electrodes are fixedly mounted on the inner surface thereof;

[0120] The signal transmission wires of the patch electrodes are hidden and fixed inside the plastic plate (in the groove between the skin-facing side and the plastic plate body);

[0121] At the connection between the plastic plates (i.e., the junction between the elastic band and the plastic plate), the signal transmission wire of the patch electrode is covered with a flexible plastic protective cover to provide stress relief and mechanical protection to prevent the wire from bending and damage;

[0122] A flexible and retractable ring-shaped wristband is formed by connecting the plastic plates with a high-elasticity elastic band. The elasticity of the elastic band enables the adjustment of the overall wearing circumference of the watch.

[0123] S2: Uses the camera module to capture the user's arm, locates the user's elbow joint based on the captured photo, and calculates the length of the user's arm;

[0124] S21: establishing an image coordinate system based on the photograph, where the coordinates of the user's elbow joint in the photograph are (picx, pici);

[0125] Convert the coordinates of the user's elbow joint in the captured photo to camera coordinates (camx, camy, camz):

[0126]

[0127] Among them, foc represents the focal length of the camera, [] -1 represents the inverse matrix;

[0128] Convert camera coordinates to world coordinates (worlx, worly, worlz):

[0129]

[0130] Among them, [Rota|Tran] represents the external parameters of the camera, which are the rotation matrix and translation vector [tranx trany tranz] T composition;

[0131] S22: Convert the world coordinates (worlx, worly, worlz) to the coordinates in the dial coordinate system (dix, diy, diz):

[0132] Based on the coordinates (wox, woy, woz) corresponding to the origin of the dial coordinate system in the world coordinate system, obtain the vector WO from the origin of the world coordinate system to the origin of the dial coordinate system: WO = (wox, woy, woz);

[0133] In the dial coordinate system, the unit vector of the x-axis in the world coordinate system is expressed as WDx = (wdialxx, wdialxy, wdialxz), the unit vector of the y-axis in the world coordinate system is expressed as WDy = (wdialyx, wdialyy, wdialyz), and the unit vector of the z-axis in the world coordinate system is expressed as WDz = (wdialzx, wdialzy, wdialzz);

[0134] Convert the world coordinates (worlx, worly, worlz) into the coordinates of the dial coordinate system (dix, diy, diz);

[0135]

[0136] S23: Calculate the length of the user's arm darm based on the coordinates in the dial coordinate system: darm=(dix 2 +DIY 2 +diz 2 ) 1 / 2 .

[0137] S3: The signal processing module filters the collected electromyographic signal to obtain a filtered electromyographic signal;

[0138] A second-order Butterworth bandpass filter is used to filter the EMG signal to retain the main frequency components of the EMG signal;

[0139] A first-order high-pass filter is used to filter the electromyographic signal to remove the DC offset and low-frequency drift in the electromyographic signal.

[0140] In the embodiment of the present invention, the transfer function H of the second-order Butterworth bandpass filter is bt (fre) is expressed as:

[0141]

[0142] Among them, fre represents the signal frequency, ω center represents the center angular frequency, Qua represents the quality factor, ω low Represents the low-frequency cutoff angular frequency, ω high Represents the high-frequency cutoff corner frequency.

[0143] S4: The filtered electromyographic signal is subjected to feature extraction through a time series convolutional network. The main control module calculates the angular velocity of the user's elbow joint based on the wrist movement speed acquired by the inertial measurement module and the arm length acquired by the camera module. The main control module triggers an early warning mechanism when the following two conditions are simultaneously met: first, the analysis result of the filtered electromyographic signal by the time series convolutional network is determined to be a fatigue state; second, the calculated angular velocity of the user's elbow joint is greater than a preset threshold.

[0144] In the embodiment of the present invention, the main control module includes a signal processing part and a display screen, and the display screen is located on the front of the dial;

[0145] The main control module extracts features from the filtered EMG signals using a time-series convolutional network. Simultaneously, the module calculates the angular velocity of the user's elbow joint based on the wrist movement speed collected by the inertial measurement module and the arm length obtained by the camera module. The main control module triggers an early warning mechanism when two conditions are met simultaneously: first, the time-series convolutional network determines fatigue based on the filtered EMG signals; second, the calculated angular velocity of the user's elbow joint exceeds a preset threshold.

[0146] It should be noted that, in the embodiment of the present invention, the temporal convolutional network is a deep learning model suitable for time series processing, which adopts an architecture design that connects causal convolution and dilated convolution in series. Among them, causal convolution ensures that the output of the current moment depends only on the elements of the previous layer at the current moment and earlier; dilated convolution constructs an effective filter of larger size by introducing zero padding or skipping part of the input in the original filter.

[0147] In the embodiment of the present invention, the filtered electromyographic signal sequence is first divided into signal segments of equal length to form an electromyographic signal matrix Filsig:

[0148]

[0149] Among them, fil_sig 1,1 ,fil_sig 1,2 ,…,fil_sig 1,snum Indicates the equal-length signal segments obtained by dividing the filtered electromyographic signal sequence with sequence number 1, fil_sig 2,1 ,fil_sig 2,2 ,…,fil_sig 2,snum It represents the equal-length signal segments obtained by dividing the filtered electromyographic signal sequence with sequence number 2, snum represents the number of divided signal segments, and tnum represents the number of filtered electromyographic signal sequences;

[0150] Output of the lay-th layer causal convolution Causelay And the output of the lay-th layer expansion convolution Expansion lay Respectively expressed as:

[0151] Cause lay =weight c ·Expan lay-1 +bias c

[0152] Expan lay =weight e Cause lay-1 ′+bias e

[0153] Among them, Expansion lay-1 represents the output of the clay-1th layer dilated convolution, weight c Represents the weight of the causal convolution kernel, weight e Represents the weight of the dilated convolution kernel, bias c Represents the bias of the causal convolution kernel, bias e Represents the bias of the dilated convolution kernel; and satisfies Expan0 = Filsig; Cause lay-1 Represents the output of the lay-1 layer causal convolution, Cause lay-1 ′ is Cause lay-1 Based on this, we introduce the expansion factor to expand it;

[0154] The angular velocity of the user's elbow joint is expressed as:

[0155] elbow=((v x (t) / darm) 2 +(v y (t) / darm) 2 ) 1 / 2 .

[0156] In the embodiment of the present invention, the power module is responsible for supplying power to the watch and controlling the output power according to whether the watch is in working mode.

[0157] Example 2: The present invention also provides an application system for a myoelectric intelligent warning watch for fatigue driving, comprising the following six modules:

[0158] Inertial measurement module: monitors the user's posture and activates the working mode when the user is in a sitting position; measures the real-time acceleration of the user's wrist and calculates the movement speed and displacement of the user's wrist; measures the angular velocity of the user's wrist and, combined with the quaternion algorithm, calculates the rotation angle of the user's wrist;

[0159] Signal acquisition module: uses patch electrodes to collect electromyographic signals;

[0160] Camera module: responsible for photographing the user's arm, locating the user's elbow joint based on the photographed image, and calculating the length of the user's arm;

[0161] Signal processing module: filters the electromyographic signals collected by the signal acquisition module;

[0162] Main control module: This module extracts features from the filtered EMG signals and calculates the angular velocity of the user's elbow joint. If the temporal convolutional network analyzes the filtered EMG signals and determines they are in a fatigue state, and the calculated angular velocity of the user's elbow joint is greater than a preset threshold, an early warning mechanism is triggered.

[0163] Power module: Provides power to the watch and controls the output power according to whether the watch is in working mode.

[0164] It should be noted that the serial numbers of the above-mentioned embodiments of the present invention are for descriptive purposes only and do not represent the advantages or disadvantages of the embodiments. In addition, the terms "including", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method comprising the element.

[0165] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0166] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An application method of an electromyographic intelligent warning watch for fatigue driving, characterized in that: The method comprises: S1: The inertial measurement module is responsible for monitoring the user's posture. When the user is in a sitting position, the working mode is activated to measure the real-time acceleration of the user's wrist and calculate the movement speed and displacement of the user's wrist. The angular velocity of the user's wrist is measured and combined with the quaternion algorithm to obtain the rotation angle of the user's wrist. The signal acquisition module begins to collect electromyographic signals using patch electrodes. S2: Uses the camera module to capture the user's arm, locates the user's elbow joint based on the captured photo, and calculates the length of the user's arm; S3: The signal processing module filters the collected electromyographic signal to obtain a filtered electromyographic signal; S4: The filtered electromyographic signal is subjected to feature extraction through a time series convolutional network. The main control module calculates the angular velocity of the user's elbow joint based on the wrist movement speed collected by the inertial measurement module and the arm length obtained by the camera module. The main control module triggers the early warning mechanism when the following two conditions are met at the same time: first, the analysis result of the filtered electromyographic signal by the time series convolutional network is judged to be a fatigue state; second, the calculated angular velocity of the user's elbow joint is greater than a preset threshold.

2. The application method of the myoelectric intelligent warning watch for fatigue driving according to claim 1 is characterized in that: The step S1 comprises: S11: The user wears the watch on the wrist, with the dial located at the center of the radiocarpal joint; With the center of the radiocarpal joint as the coordinate origin, establish a dial coordinate system. The plane passing through the center of the radiocarpal joint and parallel to the palm is the xoz plane. The line perpendicular to the xoz plane and passing through the center of the radiocarpal joint is the y-axis, with the direction from the palm to the back of the hand as the positive y-axis direction. The direction parallel to the xoz plane, passing through the center of the radiocarpal joint, and from the wrist near the little finger to the wrist near the thumb is set as the x-axis direction. The direction parallel to the xoz plane, passing through the center of the radiocarpal joint, and from the elbow to the fingers is set as the z-axis direction. S12: The inertial measurement module measures the real-time acceleration of the user's wrist and calculates the speed and displacement of the user's wrist. Combined with the quaternion algorithm, the rotation angle of the user's wrist is obtained. S13: When the absolute value of the movement displacement of the user's wrist is less than the activity threshold, and the absolute value of the rotation angle of the user's wrist is less than the standing threshold, it is determined that the user is in a sitting state.

3. The application method of the myoelectric intelligent warning watch for fatigue driving according to claim 2 is characterized in that: The step S12 includes: The velocity vector v(t) of the user's wrist at time t is expressed as: Where v(t)=[v x (t),v y (t),v z (t)],v x (t),v y (t),v z (t) represents the movement speed of the user's wrist in the x, y, and z directions at time t, respectively, and acce(τ) represents the movement acceleration vector of the user's wrist at time τ; acce(τ)=[a x (t),a y (t),a z (τ)-pro(τ)·g] Among them, a x (τ),a y (τ),a z (τ) represents the acceleration of the user's wrist in the x, y, and z directions at time τ, measured by the inertial measurement module; pro(τ) represents the acceleration due to gravity to a z (τ) direction projection, g represents the acceleration of gravity; at time 0, the user puts his arm vertically to initialize the watch, at this time pro(0) = 1; The displacement vector dis(t) of the user's wrist at time t is expressed as: Where dis(t)=[d x (t),dy(t),d z (t)],d x (t),d y (t),d z (t) represents the displacement of the user's wrist in the x, y, and z directions at time t, respectively; When t=T, T represents the time period, and the absolute value of the user's wrist displacement abdis is expressed as: abdis=((d x (THE)) 2 +(d y (THE)) 2 +(d z (THE)) 2 ) 1 / 2 Among them, d x (T), d y (T), d z (T) represents the displacement of the user's wrist in the x, y, and z directions at time T, respectively; At the initial moment, according to the initial posture, determine the initial posture quaternion q(0): q(0) = [1,0,0,0] T ,in[] T represents the transposed matrix; The angular velocity vector ω(t) of the user's wrist at time t is expressed as: ω(t)=[ω x (t),ω y (t),ω z (t)] Among them, ω x (t),ω y (t),ω z (t) represents the angular velocity of the user's wrist in the x, y, and z directions at time t; the posture quaternion q(t) at time t = [q ω (t),q x (t),q y (t),qz(t)] T , satisfying (q ω (t)) 2 +(q x (t)) 2 +(q y (t)) 2 +(q z (t)) 2 =1, where q ω (t) represents the real part of the attitude quaternion at time t, q x (t) represents the component of the imaginary part x of the attitude quaternion at time t, q y (t) represents the component of the imaginary part y of the attitude quaternion at time t, q z (t) represents the component of the imaginary part z of the attitude quaternion at time t, specifically: allang(t)=((ω x (t-1)+ω x (t)·Δt) 2 +(ω y (t-1)+ω y (t)·Δt) 2 +(ω z (t-1)+ω z (t)·Δt) 2 ) 1 / 2 Among them, allang(t) represents the rotation angle of the user's wrist at time t, ω x (t-1),ω y (t-1),ω z (t-1) represents the angular velocity of the user's wrist in the x, y, and z directions at time t-1, Δt represents the time sampling interval, sin represents the sine function, and cos represents the cosine function; The instantaneous rate of change of the attitude quaternion at time t Expressed as: in, The quaternion ω representing the angular velocity of the attitude quaternion q(t) at time t q (t) The instantaneous rate of change under the action of represents quaternion multiplication, q(t+1) represents the attitude quaternion at time t+1, ω q (t) represents the quaternion representation of the angular velocity at time t, ω q (t)=[0,ω x (t),ωy(t),ω z (t)] T ; When t=T, the rotation angle of the user's wrist is expressed as: Where θ represents the pitch angle of the user's wrist rotation, represents the roll angle of the user's wrist rotation, ψ represents the yaw angle of the user's wrist rotation, and q ω (T) represents the real part of the attitude quaternion at time T, q x (T) represents the component of the imaginary part x of the attitude quaternion at time T, q y (T) represents the component of the imaginary part y of the attitude quaternion at time T, q z (T) represents the component of the imaginary part z of the attitude quaternion at time T; The absolute value of the user's wrist rotation angle abangle is expressed as:

4. The application method of the myoelectric intelligent warning watch for fatigue driving according to claim 1 is characterized in that: The step S1 further includes: The signal acquisition module includes patch electrodes and an adjustable wristband assembly; The adjustable wristband assembly is composed of a plurality of plastic plates connected by a high-elasticity elastic band. When worn, the plastic plates contact the skin, and patch electrodes are fixedly mounted on the inner surfaces of the plastic plates. The plastic plates are rigid or semi-rigid plastic plates. The signal transmission wire of the patch electrode is hidden and fixed in the groove between the skin-facing side of the plastic plate and the plastic plate; At the connection between the plastic plates, the signal transmission wires of the patch electrodes are covered with a flexible plastic protective cover; A flexible and retractable ring-shaped wristband is formed by connecting the plastic plates with a high-elasticity elastic band. The elasticity of the elastic band enables the adjustment of the overall wearing circumference of the watch.

5. The application method of the myoelectric intelligent warning watch for fatigue driving according to claim 1 is characterized in that: The step S2 comprises: S21: establishing an image coordinate system based on the photograph, where the coordinates of the user's elbow joint in the photograph are (picx, pici); Convert the coordinates of the user's elbow joint to camera coordinates (camx, camy, camz): Among them, foc represents the focal length of the camera, [] -1 represents the inverse matrix; Convert camera coordinates to world coordinates (worlx, worly, worlz): Among them, [Rota|Tran] represents the external parameters of the camera, which are the rotation matrix and translation vector [tranx trany tranz] T composition; S22: Convert the world coordinates (worlx, worly, worlz) into the coordinates of the dial coordinate system (dix, diy, diz); S23: Calculate the user's arm length darm based on the coordinates in the dial coordinate system: darm=(dix 2 +DIY 2 +diz 2 ) 1 / 2 .

6. The application method of the myoelectric intelligent warning watch for fatigue driving according to claim 5 is characterized in that: The step S22 includes: Based on the coordinates (wox, woy, woz) corresponding to the origin of the dial coordinate system in the world coordinate system, obtain the vector WO from the origin of the world coordinate system to the origin of the dial coordinate system: WO = (wox, woy, woz); In the dial coordinate system, the unit vector of the x-axis in the world coordinate system is expressed as WDx = (wdialxx, wdialxy, wdialxz), the unit vector of the y-axis in the world coordinate system is expressed as WDy = (wdialyx, wdialyy, wdialyz), and the unit vector of the z-axis in the world coordinate system is expressed as WDz = (wdialzx, wdialzy, wdialzz); Convert the world coordinates (worlx, worly, worlz) into the coordinates of the dial coordinate system (dix, diy, diz); 7. The application method of the myoelectric intelligent warning watch for fatigue driving according to claim 1 is characterized in that: The step S3 comprises: A second-order Butterworth bandpass filter is used to filter the EMG signal to retain the main frequency components of the EMG signal; A first-order high-pass filter is used to filter the electromyographic signal to remove the DC offset and low-frequency drift in the electromyographic signal; Obtain the filtered electromyographic signal.

8. An application system for myoelectric intelligent warning watches for fatigue driving, characterized by: include: Inertial measurement module: monitors the user's posture and starts working mode when the user is in a sitting position; Measure the real-time acceleration of the user's wrist and calculate the speed and displacement of the user's wrist; measure the angular velocity of the user's wrist and, combined with the quaternion algorithm, obtain the rotation angle of the user's wrist; Signal acquisition module: uses patch electrodes to collect electromyographic signals; Camera module: responsible for photographing the user's arm, locating the user's elbow joint based on the photographed image, and calculating the length of the user's arm; Signal processing module: filters the electromyographic signals collected by the signal acquisition module; Main control module: extracts features from the filtered electromyographic signals and calculates the angular velocity of the user's elbow joint; When the temporal convolutional network analyzes the filtered electromyographic signal and determines it is in a fatigue state, and the calculated angular velocity of the user's elbow joint is greater than the preset threshold, the warning mechanism is triggered; Power module: supplies power to the watch and controls the output power according to whether the watch is in working mode; To realize a myoelectric intelligent warning watch for fatigue driving as described in any one of claims 1-7.

Images