Fatigue state detection method, system and device and storage medium

Abstract

The invention discloses a fatigue state detection method, system and device and a storage medium, and relates to the technical field of fatigue state detection.The method comprises the steps that an electroencephalogram signal, an electrocardiosignal, a respiration electric signal, an abdominal respiration curve, respiration amplitude and an eye movement image of a user are collected; determining an electroencephalogram fatigue index, an electrocardio fatigue index, a respiratory fatigue index and an eye movement fatigue index of the user; taking the electroencephalogram fatigue index, the electrocardio fatigue index, the respiratory fatigue index and the eye movement fatigue index as evidences, and calculating basic probability assignment (BPA) of the evidences corresponding to different fatigue degrees of the user; the D-S evidence theory is used for conducting decision-level fusion on the electroencephalogram evidence BPA, the electrocardiogram evidence BPA, the respiration evidence BPA and the eye movement evidence BPA, the comprehensive fatigue index of the user is calculated, and therefore the fatigue state of the user is detected; according to the method, the dependence of a single physiological signal is reduced, and the detection accuracy of the fatigue state of the pilot is improved.

Description

Technical Field

[0001] This invention relates to the field of fatigue detection technology, specifically to a fatigue state detection method, system, device, and storage medium. Background Technology

[0002] With technological advancements and improvements in aircraft automation and management, the proportion of flight accidents caused by mechanical failures is gradually decreasing, while the proportion of accidents due to pilot error remains high. According to statistics from the International Civil Aviation Organization (ICAO), human factors account for as much as 76% of modern aviation accidents, with over 60% caused by pilot error and 21% related to pilot fatigue. Therefore, real-time fatigue monitoring of pilots, assessing their fatigue status, and providing alerts and warnings are crucial for reducing flight accidents caused by pilot error due to fatigue and are of great significance to aviation safety.

[0003] While some progress has been made in detecting pilot fatigue, the collection and utilization of pilot physiological indicators have not been achieved, or the physiological signals used are relatively limited. Patent application CN201611001195.7 proposes a pilot fatigue detection system and method based on the movement of aircraft control equipment. This invention couples two sensors to the aircraft control equipment and the fuselage, respectively detecting the frequency and intensity of their movement to determine the pilot's fatigue state and duration. It is non-intrusive and relatively inexpensive, but it does not detect the pilot themselves, allowing only a preliminary, rough assessment with relatively low accuracy. Patent application CN201711478611.7 proposes a pilot fatigue detection device based on electroencephalogram (EEG) signals. This invention collects and processes the pilot's EEG signals, preserving their abstract characteristics to determine fatigue levels. However, it uses only a single type of EEG information, making it highly dependent on EEG data and lacking comprehensive detection capabilities. Summary of the Invention

[0004] To address the shortcomings of existing technologies that typically rely on only a single or limited amount of physiological information, resulting in incomplete and inaccurate detection of user fatigue, this invention proposes a fatigue state detection method, system, device, and storage medium. By integrating multi-source physiological information collected from users at the decision-making level and then making a comprehensive judgment, the problems existing in the prior art are solved.

[0005] A fatigue state detection method includes the following steps: Collect users' physiological signal data; the physiological signal data includes electroencephalogram (EEG) signals, electrocardiogram (ECG) signals, respiratory electrical signals, and eye movement images; The user's EEG fatigue index was determined by extracting energy components from the Delta, Theta, Alpha, Beta, and Gamma bands of the EEG signal; the user's ECG fatigue index was determined by performing Fast Fourier Transform and power spectrum analysis on the ECG signal; the user's respiratory fatigue index was calculated by collecting peak extreme points from the respiratory electrical signal; and the user's eye movement fatigue index was determined based on the fixation duration of blinking and the duration of eye closure in the eye movement images. Using EEG fatigue index, ECG fatigue index, respiratory fatigue index, and eye movement fatigue index as evidence, the basic probability allocation (BPA) for each piece of evidence corresponding to different levels of user fatigue was calculated, resulting in EEG evidence BPA, ECG evidence BPA, respiratory evidence BPA, and eye movement evidence BPA. DS evidence theory was used to perform decision-level fusion of EEG evidence BPA, ECG evidence BPA, respiratory evidence BPA, and eye movement evidence BPA to calculate the user's comprehensive fatigue index. The user's fatigue status is detected based on the comprehensive fatigue index.

[0006] Furthermore, the step of determining the user's brain fatigue index by extracting components of the EEG signal in the Delta, Theta, Alpha, Beta, and Gamma bands specifically includes the following steps: The energy components of EEG signals in the Delta, Theta, Alpha, Beta, and Gamma bands were extracted using Fast Fourier Transform and Power Spectrum Analysis. , , , and ; By calculating the proportions of the Delta and Theta bands 100 indicates the user's brain fatigue index.

[0007] Furthermore, the step of determining the user's cardiac fatigue index by performing Fast Fourier Transform and power spectral analysis on the electrocardiogram signal specifically includes the following steps: Fast Fourier Transform and power spectrum analysis were performed on the electrocardiogram (ECG) signal to obtain the ECG power spectrum and power spectral density. The high-frequency and low-frequency components of the power spectral density of the electrocardiogram (ECG) signal are separated, and the power ratio of the high-frequency and low-frequency components and the ratio of low-frequency to high-frequency components of the ECG signal are calculated. Mark the R-peaks in the electrocardiogram signal and store the corresponding time of the R-peaks. Record the number of R-peaks per unit time to obtain the user's heart rate. Calculate the time difference between two adjacent R peaks to obtain the RR interval value; and calculate the standard deviation of the RR interval value within 30 seconds before that time. Based on the mean, standard deviation, root mean square of the heart rate and RR interval, as well as the power ratio of the high and low frequency components of the ECG signal and the ratio of low frequency to high frequency, the user's ECG fatigue index is obtained.

[0008] Furthermore, the step of calculating the user's respiratory fatigue index by collecting the peak extreme points of the respiratory electrical signal specifically includes the following steps: Collect the peaks and troughs of the respiratory electrical signal and their corresponding times, record the number of peaks or troughs of the respiratory electrical signal per unit time, and calculate the frequency of the respiratory signal; calculate the standard deviation of the respiratory frequency based on the respiratory frequencies of the previous 5 respiratory times at that time. The user's respiratory amplitude data is calculated based on the difference in displacement data at corresponding times of adjacent peaks and troughs; the standard deviation of the respiratory amplitude is then calculated based on the respiratory amplitude data. The respiratory fatigue index of the user is determined based on the frequency, frequency standard deviation, and respiratory amplitude standard deviation of the respiratory signal, as well as the user's abdominal breathing curve.

[0009] Furthermore, the eye fatigue index of the user is determined based on the fixation duration and eye closure duration of blinking movements in the eye movement image; specifically, this includes the following steps: The blink frequency is calculated based on the fixation duration and eye-closing duration of blinking movements in the eye movement images. Identify the pupil region in an image of eye movement when the eyes are open, determine the number of pixels in the pupil region, and calculate the average pupil area when the user's eyes are open; The user's eye fatigue index was determined based on blink frequency, fixation duration, and average pupil area.

[0010] Furthermore, the step of using brainwave fatigue index, cardiac fatigue index, respiratory fatigue index, and eye movement fatigue index as evidence to calculate the basic probability allocation (BPA) for different levels of user fatigue for each piece of evidence includes the following steps: Define the recognition framework ,in T This indicates pilot fatigue. NT This indicates that the pilot is not fatigued; Identify the power of the frame The hypothesis space consists of four elements: empty set, user fatigue, user non-fatigue, and user fatigue or non-fatigue. Take the interval boundary , Using the brain fatigue index, cardiac fatigue index, respiratory fatigue index, and eye movement fatigue index as evidence, the basic probability assignment (BPA) of each element in the hypothesis space for each piece of evidence is calculated. The calculation process is as follows: For the brain fatigue index, a normal distribution is constructed. The BPA for calculating EEG evidence is: For the cardiac fatigue index, the BPA for calculating EEG evidence is: ; For the respiratory fatigue index, the BPA for calculating EEG evidence is: ; For the eye movement fatigue index, the BPA for calculating EEG evidence is: .

[0011] Furthermore, the step of using DS evidence theory to perform decision-level fusion of EEG evidence BPA, ECG evidence BPA, respiratory evidence BPA, and eye movement evidence BPA to calculate the user's comprehensive fatigue index specifically includes the following steps: The normalization constant is calculated as follows: According to the DS evidence theory, we get: The composition probability distribution of each focal element in the hypothesis space is calculated as follows: , and ; Substituting the composite probability allocation into the BPA expression, the user's comprehensive fatigue index is calculated. .

[0012] The present invention also includes a fatigue condition detection system, comprising: The acquisition module is used to acquire the user's physiological signal data, including electroencephalogram (EEG) signals, electrocardiogram (ECG) signals, respiratory electrical signals, and eye movement images. The index determination module is used to determine the user's brain fatigue index by extracting the energy components of the EEG signal in the Delta, Theta, Alpha, Beta, and Gamma bands; to determine the user's cardiac fatigue index by performing Fast Fourier Transform and power spectrum analysis on the ECG signal; to calculate the user's respiratory fatigue index by collecting the peak extreme points in the respiratory electrical signal; and to determine the user's eye movement fatigue index based on the fixation duration and eye closure duration of blinking movements in the eye movement image. The fusion module is used to use EEG fatigue index, ECG fatigue index, respiratory fatigue index, and eye movement fatigue index as evidence to calculate the basic probability allocation (BPA) for each piece of evidence corresponding to different levels of user fatigue, resulting in EEG evidence BPA, ECG evidence BPA, respiratory evidence BPA, and eye movement evidence BPA. DS evidence theory is then used to perform decision-level fusion of the EEG evidence BPA, ECG evidence BPA, respiratory evidence BPA, and eye movement evidence BPA to calculate the user's comprehensive fatigue index. The fatigue detection module is used to detect the user's fatigue status based on the comprehensive fatigue index.

[0013] The present invention also includes a fatigue state detection computer device, characterized in that it includes: a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the fatigue state detection method.

[0014] The present invention also includes a readable storage medium, characterized in that the readable storage medium stores a computer program, the computer program including program instructions, which, when executed by a processor, are used to perform the steps of the fatigue state detection method.

[0015] This invention provides a fatigue state detection method, which has the following beneficial effects: This invention reduces reliance on single physiological indicators by real-time monitoring of pilots' EEG, ECG, respiration, and eye movement (EMG) indicators. It collects multi-source physiological information from users and uses the DS evidence theory to perform decision-level fusion of EEG, ECG, respiration, and EMG evidence (BPA). This allows for explicit representation and preservation of uncertainties in each modality of evidence, providing a comprehensive assessment of fatigue status based on EEG, ECG, respiration, and EMG signals. It exhibits good real-time performance and fault tolerance, enabling the integration of multi-dimensional physiological information during flight to assess pilot fatigue levels and reduce the likelihood of flight accidents. Compared to existing technologies, this method reduces reliance on single physiological signals and improves the accuracy of pilot fatigue detection. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the overall structure of a fatigue detection device based on the fusion of multiple physiological information in an embodiment of the present invention; Figure 2 This is a schematic diagram of the headgear used for collecting physiological indicators in an embodiment of the present invention; Figure 3 This is a schematic diagram of the vest used for collecting physiological indicators in an embodiment of the present invention; Figure 4 This is a flowchart illustrating the operation of the fatigue detection device in an embodiment of the present invention.

[0017] In the diagram: 1- Physiological index collection headgear; 11- EEG index collector; 2- Physiological index collection vest; 21- ECG index collector; 22- Respiratory index collector; 3- Helmet; 31- Camera device; 32- Speaker; 4- Fatigue detection chassis; 51- Head corresponding acupoint stimulation device; 52- Back corresponding acupoint stimulation device. Detailed Implementation

[0018] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0019] This invention proposes a fatigue state detection method, comprising the following steps: S1. Collect the user's physiological data; the physiological data includes electroencephalogram (EEG) signals, electrocardiogram (ECG) signals, respiratory electrical signals, abdominal breathing curves, respiratory amplitude, and eye movement images.

[0020] See Figures 1-3 As shown, this invention provides a fatigue detection device for collecting users' physiological data, including a physiological index collection headgear 1 made of elastic fabric. The headgear 1 has an EEG data acquisition device 11 positioned at the top of the pilot's head and a corresponding acupoint stimulation device 51 at the pilot's temples. The pilot wears a helmet 3 over the headgear 1. A camera 31 for detecting eye movement indicators is installed along the lower front edge of the helmet 3, and a speaker 32 for playing fatigue warning voice messages is installed at the pilot's ear. The pilot wears a physiological index collection vest 2 made of elastic fabric. The vest has an ECG data acquisition device 21 positioned at the pilot's chest, a respiratory data acquisition device 22 positioned at the pilot's abdomen, and a corresponding acupoint stimulation device 52 on the pilot's back. A fatigue detection housing 4 is installed under the pilot's seat, containing a processor and a power supply.

[0021] Using actual collected individual physiological data as an example, fatigue detection was performed using the fatigue detection device provided by this invention. The subject was a 27-year-old male. The collected physiological indicators included electroencephalogram (EEG), electrocardiogram (ECG), respiration, and eye movement (EMG). The acquisition frequency for EEG and ECG signals was 256 Hz, the acquisition frequency for respiration was 2 Hz, and the acquisition frequency for EMG signals was 24 Hz. The total acquisition time was 20 minutes, with one detection cycle every 10 seconds. The tester first applies a small amount of conductive gel to the corresponding positions of the EEG, ECG, and respiration index acquisition devices 11, 21, and 22. Then, the tester puts on a physiological index acquisition headgear 1 and vest 2, followed by a helmet 3. The tester then manually powers on the fatigue detection chassis 4 and connects the EEG, ECG, and respiration index acquisition devices 11, 21, and 22, as well as the eye movement detection camera 31, to the chassis 4 via cables. The chassis 4 is then connected to a speaker 32, a head acupoint stimulation device 51, and a back acupoint stimulation device 52 via cables. Once the fatigue detection chassis 4 detects that the detection equipment and stimulation devices are correctly connected, the processor supplies power to the EEG, ECG, respiration, and eye movement detection cameras 31, enabling them to begin operation. The system monitors the tester's EEG, ECG, respiration, and eye movement indices in real time and transmits the data to the processor via cables. The processor then determines whether the tester is fatigued based on these multiple physiological indicators. When the test subject is detected to be fatigued, the processor controls the speaker 32 to provide a voice reminder; when the test subject is detected to be in a state of severe fatigue, the processor controls the speaker 32 to provide a voice reminder and controls the stimulation device 5 to stimulate the corresponding acupoints of the test subject; at the same time, the processor stores the received fatigue indices and comprehensive fatigue index and uploads them to the server at certain intervals. After the experiment ends, the fatigue detection chassis 4 stops the collection and detection of multiple physiological indicators, and stops supplying power to the EEG index collector 11, ECG index collector 21, respiratory index collector 22, and the camera device 31 that detects eye movement indicators, so that they stop working; at the same time, the processor uploads the historical data of various fatigue indices, comprehensive fatigue index, voice reminders, and stimulation stored during the collection process to the server, clears the data stored in the processor, and shuts down the power.

[0022] S2. Calculate the fatigue index.

[0023] The process of determining the fatigue state of a tester using fatigue detection equipment is as follows: The tester undergoes fatigue index threshold adaptation; the fatigue threshold for this tester is 80, and the high fatigue threshold is 95. The fatigue detection chassis 4 includes a processor and a power supply. The processor receives multiple physiological indicators collected by the EEG indicator acquisition unit 11, the ECG indicator acquisition unit 21, the respiratory indicator acquisition unit 22, and the camera device 31 that detects eye movement indicators. It processes the data and obtains the fatigue index for each physiological indicator. In one detection cycle, the components of the EEG signal in the Delta, Theta, Alpha, Beta, and Gamma bands are extracted from the EEG indicators to determine the tester's EEG fatigue index. ;

[0024] In one detection cycle, the heart rate, mean of the RR interval, standard deviation of the RR interval, root mean square of the RR interval difference, standardized low-frequency component, standardized high-frequency component, and low-frequency to high-frequency ratio are extracted from the electrocardiogram signal to determine the test subject's electrocardiogram fatigue index. 60; Respiratory rate, standard deviation of respiratory rate, length of abdominal breathing curve, and standard deviation of respiratory amplitude are extracted from respiratory indicators to determine the respiratory fatigue index of the test subject. Blink frequency, fixation duration, and average pupil area were extracted from eye movement indicators to determine the eye fatigue index of the test subjects. .

[0025] Specifically, the process includes the following: EEG signals were extracted using Fast Fourier Transform and Power Spectrum Analysis, and the energy components in the Delta, Theta, Alpha, Beta, and Gamma bands were calculated. , , , and Calculate the proportion of Delta and Theta bands. 100 determines the user's brain fatigue index.

[0026] Mark the R-peaks in the ECG signal, calculate and store the corresponding timestamps of the R-peaks. Record the number of R-peaks within a unit of time (usually 10 seconds) to obtain the user's heart rate; calculate the difference between two adjacent R-peak times to obtain the RR interval, and calculate the standard deviation of the RR interval within the 30 seconds prior to that time. Fast Fourier Transform and power spectrum analysis were performed on the electrocardiogram (ECG) signal to obtain the power spectrum and power spectral density. The high-frequency and low-frequency components of the ECG power spectral density were separated, and the power of the high-frequency and low-frequency components of the ECG signal was calculated. and Based on the standard deviation of the obtained heart rate (HR) and heart rate (RR) intervals. Calculate the ratio of high-frequency power to low-frequency power and the ratio of low-frequency power to high-frequency power. Determine the user's electrocardiogram fatigue index.

[0027] Collect the peaks and troughs of the respiratory electrical signal and their corresponding times, record the number of peaks (or troughs) of the respiratory electrical signal per unit time (usually 10 seconds), and calculate the frequency of the respiratory signal; based on the respiratory frequencies of the previous 5 breaths at that time... ,according to Calculate the standard deviation of respiratory rate; calculate respiratory amplitude data based on the difference in displacement data at adjacent peaks and troughs of the abdominal respiratory index acquisition device, and then... Calculate the standard deviation of respiratory amplitude; based on respiratory rate. (times / min), frequency standard deviation (breaths / min), respiratory amplitude (cm) and standard deviation of respiratory amplitude (cm), calculate Determine the user's respiratory fatigue index.

[0028] Based on the fixation duration and eye-closing duration of blinking movements in eye movement images, the blink frequency is calculated; the pupil region when the eyes are open is identified in the eye movement images, the number of pixels in the pupil region is determined, and the average and maximum pupil area of ​​the user when the eyes are open are calculated; based on the blink frequency... (times / min), fixation duration t (s), and average pupil area (px) are used to calculate Determine the user's eye strain index.

[0029] S3. Using EEG fatigue index, ECG fatigue index, respiratory fatigue index, and eye movement fatigue index as evidence, calculate the basic probability allocation (BPA) for each piece of evidence corresponding to different levels of user fatigue. Use DS evidence theory to perform decision-level fusion of the EEG evidence BPA, ECG evidence BPA, respiratory evidence BPA, and eye movement evidence BPA to calculate the user's comprehensive fatigue index.

[0030] Define the recognition framework ,in T This indicates pilot fatigue. NT This indicates that the pilot is not fatigued, and the power of the recognition frame. The hypothesis space consists of four elements, representing four scenarios: an empty set, pilot fatigue, pilot non-fatigue, and pilot fatigue or non-fatigue (i.e., it cannot be determined).

[0031] Take the interval boundary , .

[0032] Using the EEG fatigue index, ECG fatigue index, respiratory fatigue index, and eye movement fatigue index as evidence 1 to 4, the basic probability assignment (BPA) of each evidence element in the hypothesis space was calculated.

[0033] For evidence 1—the EEG fatigue index—a normal distribution is constructed. ,Right now The BPA for calculating EEG evidence is: Using a similar normal distribution , and The basic probability allocation (BPA) for Evidence 2 – ECG fatigue index, Evidence 3 – respiratory fatigue index, and Evidence 4 – eye movement fatigue index can be calculated as follows: Ignoring the empty set portion, the BPA of the aforementioned EEG, ECG, respiration, and eye movement evidence is merged to form multi-physiological information fusion evidence.

[0034] Table 1. BPA obtained by testers using this method The process of using the DS evidence theory to perform decision-level fusion of EEG evidence (BPA), ECG evidence (BPA), respiratory evidence (BPA), and eye movement evidence (BPA) to calculate the pilot's comprehensive fatigue index includes: The normalization constant is calculated as follows: According to the DS evidence theory, the probability distribution of synthesis for each focal element in the hypothesis space is calculated as follows: Substituting the composition probability allocation back into the aforementioned BPA calculation formula, we obtain the normal distribution that satisfies the following: Therefore, in this testing cycle, the overall fatigue index of the testers is the expected value of this normal distribution, i.e. 86.

[0035] The comprehensive fatigue index is used to determine and store the fatigue state. In five consecutive testing cycles, the test subject's comprehensive fatigue index was 77, 82, 86, 87, and 84, respectively. Since the comprehensive fatigue index exceeded the preset fatigue threshold for three consecutive testing cycles, the test subject was determined to be in a fatigued state. The processor sent a signal to speaker 32, which then issued a fatigue voice alert. In another five consecutive testing cycles, the test subject's comprehensive fatigue index was 86, 91, 95, 96, and 97, respectively. Since the comprehensive fatigue index exceeded the preset high fatigue threshold for three consecutive testing cycles, the test subject was determined to be in a high fatigue state. The processor sent signals to speaker 32, the head acupoint stimulation device 51, and the back acupoint stimulation device 52. Upon receiving the signals, speaker 32 issued fatigue and stimulation voice alerts. The head and back acupoint stimulation devices 51 and 52, upon receiving the signals, discharged electrical stimulation to the corresponding acupoints on the head and back to reduce the test subject's fatigue level.

[0036] S4. Detect the user's fatigue status based on the comprehensive fatigue index.

[0037] In summary, this invention provides a fatigue detection method based on the fusion of multiple physiological information. By real-time monitoring of pilots' electroencephalogram (EEG), electrocardiogram (ECG), respiration, and eye movements, it reduces reliance on single physiological indicators. It can integrate multiple physiological information during flight to assess the pilot's fatigue index, determine the pilot's fatigue level, and apply different alerts and stimuli to different fatigue states, enabling the pilot to remain alert for longer periods during flight and reducing the likelihood of flight accidents. Furthermore, the multiple physiological indicators and fatigue states of the pilot during flight are stored, facilitating post-flight analysis and targeted training.

[0038] Based on the same inventive concept, this invention also proposes a fatigue state detection system, comprising: The acquisition module is used to collect the user's physiological signal data, including electroencephalogram (EEG) signals, electrocardiogram (ECG) signals, respiratory electrical signals, and eye movement images.

[0039] The index determination module is used to determine the user's EEG fatigue index by extracting energy components from the Delta, Theta, Alpha, Beta, and Gamma bands of the EEG signal; to determine the user's ECG fatigue index by performing Fast Fourier Transform and power spectrum analysis on the ECG signal; to calculate the user's respiratory fatigue index by collecting peak extreme points in the respiratory electrical signal; and to determine the user's eye movement fatigue index based on the fixation duration and eye closure duration of blinking movements in the eye movement image.

[0040] The fusion module is used to use EEG fatigue index, ECG fatigue index, respiratory fatigue index, and eye movement fatigue index as evidence to calculate the basic probability allocation (BPA) for each piece of evidence corresponding to different levels of user fatigue, resulting in EEG evidence BPA, ECG evidence BPA, respiratory evidence BPA, and eye movement evidence BPA. DS evidence theory is then used to perform decision-level fusion of the EEG evidence BPA, ECG evidence BPA, respiratory evidence BPA, and eye movement evidence BPA to calculate the user's comprehensive fatigue index. The fatigue detection module is used to detect the user's fatigue status based on the comprehensive fatigue index.

[0041] Based on the same inventive concept, the present invention also proposes a fatigue state detection computer device, comprising: a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the fatigue state detection method.

[0042] Based on the same inventive concept, the present invention also proposes a readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, are used to perform the steps of a fatigue state detection method.

[0043] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A fatigue state detection method, characterized in that, Includes the following steps: Collect users' physiological signal data; The physiological signal data includes electroencephalogram (EEG) signals, electrocardiogram (ECG) signals, respiratory electrical signals, and images of eye movements; The user's brain fatigue index was determined by extracting the energy components of the EEG signal in the Delta, Theta, Alpha, Beta, and Gamma bands; the user's cardiac fatigue index was determined by performing fast Fourier transform and power spectrum analysis on the ECG signal. The respiratory fatigue index of the user was calculated by collecting the peak and extreme points of the respiratory electrical signal; the eye movement fatigue index of the user was determined based on the fixation duration of blinking and the duration of eye closure in the eye movement image. Using EEG fatigue index, ECG fatigue index, respiratory fatigue index, and eye movement fatigue index as evidence, the basic probability allocation (BPA) for each piece of evidence corresponding to different levels of user fatigue was calculated, resulting in EEG evidence BPA, ECG evidence BPA, respiratory evidence BPA, and eye movement evidence BPA. DS evidence theory was used to perform decision-level fusion of EEG evidence BPA, ECG evidence BPA, respiratory evidence BPA, and eye movement evidence BPA to calculate the user's comprehensive fatigue index. The user's fatigue status is assessed based on the comprehensive fatigue index.

2. The fatigue state detection method according to claim 1, characterized in that, The method of determining the user's brain fatigue index by extracting components of the EEG signal in the Delta, Theta, Alpha, Beta, and Gamma bands specifically includes the following steps: The energy components of EEG signals in the Delta, Theta, Alpha, Beta, and Gamma bands were extracted using Fast Fourier Transform and Power Spectrum Analysis. , , , and ; By calculating the proportions of the Delta and Theta frequency bands 100 indicates the user's brain fatigue index.

3. The fatigue state detection method according to claim 1, characterized in that, The process of determining the user's cardiac fatigue index by performing Fast Fourier Transform and power spectrum analysis on the electrocardiogram signal includes the following steps: Fast Fourier Transform and power spectrum analysis were performed on the electrocardiogram (ECG) signal to obtain the ECG power spectrum and power spectral density. The high-frequency and low-frequency components of the power spectral density of the electrocardiogram (ECG) signal are separated, and the power ratio of the high-frequency and low-frequency components and the ratio of low-frequency to high-frequency components of the ECG signal are calculated. Mark the R-peaks in the electrocardiogram signal and store the corresponding time of the R-peaks. Record the number of R-peaks per unit time to obtain the user's heart rate. Calculate the time difference between two adjacent R peaks to obtain the RR interval value; and calculate the standard deviation of the RR interval value within 30 seconds before that time. Based on the mean, standard deviation, root mean square of the heart rate and RR interval, as well as the power ratio of the high and low frequency components of the ECG signal and the ratio of low frequency to high frequency, the user's ECG fatigue index is obtained.

4. The fatigue state detection method according to claim 1, characterized in that, The process of calculating the user's respiratory fatigue index by collecting peak extreme points in the respiratory electrical signal specifically includes the following steps: Collect the peaks and troughs of the respiratory electrical signal and their corresponding times, record the number of peaks or troughs of the respiratory electrical signal per unit time, and calculate the frequency of the respiratory signal; calculate the standard deviation of the respiratory frequency based on the respiratory frequencies of the previous 5 respiratory times at that time. The user's respiratory amplitude data is calculated based on the difference in displacement data at corresponding times of adjacent peaks and troughs; the standard deviation of the respiratory amplitude is then calculated based on the respiratory amplitude data. The respiratory fatigue index of the user is determined based on the frequency, frequency standard deviation, and respiratory amplitude standard deviation of the respiratory signal, as well as the user's abdominal breathing curve.

5. The fatigue state detection method according to claim 1, characterized in that, The eye fatigue index of the user is determined based on the fixation duration and eye closure duration of blinking movements in the eye movement images; specifically, it includes the following steps: The blink frequency is calculated based on the fixation duration and eye-closing duration of blinking movements in the eye movement images. Identify the pupil region in an image of eye movement when the eyes are open, determine the number of pixels in the pupil region, and calculate the average pupil area of ​​the user when the eyes are open; The user's eye fatigue index was determined based on blink frequency, fixation duration, and average pupil area.

6. The fatigue state detection method according to claim 1, characterized in that, The method of using brainwave fatigue index, cardiac fatigue index, respiratory fatigue index, and eye movement fatigue index as evidence to calculate the basic probability allocation (BPA) for different levels of user fatigue for each piece of evidence includes the following steps: Define the recognition framework ,in T This indicates pilot fatigue. NT This indicates that the pilot is not fatigued; Identify the power of the frame The hypothesis space consists of four elements: empty set, user fatigue, user non-fatigue, and user fatigue or non-fatigue. Take the interval boundary , Using the brain fatigue index, cardiac fatigue index, respiratory fatigue index, and eye movement fatigue index as evidence, the basic probability assignment (BPA) of each element in the hypothesis space for each piece of evidence is calculated. The calculation process is as follows: For the brain fatigue index, a normal distribution is constructed. The BPA for calculating EEG evidence is: For the cardiac fatigue index, the BPA for calculating EEG evidence is: ; For the respiratory fatigue index, the BPA for calculating EEG evidence is: ; For the eye movement fatigue index, the BPA for calculating EEG evidence is: .

7. The fatigue state detection method according to claim 6, characterized in that, The method of using DS evidence theory to perform decision-level fusion of EEG evidence BPA, ECG evidence BPA, respiratory evidence BPA, and eye movement evidence BPA to calculate the user's comprehensive fatigue index includes the following steps: The normalization constant is calculated as follows: According to the DS evidence theory, we get: The composition probability distribution of each focal element in the hypothesis space is calculated as follows: , and ; Substituting the composite probability allocation into the BPA expression, the user's comprehensive fatigue index is calculated. .

8. A fatigue condition detection system, characterized in that, include: The acquisition module is used to collect the user's physiological signal data; The physiological signal data includes electroencephalogram (EEG) signals, electrocardiogram (ECG) signals, respiratory electrical signals, and images of eye movements; The index determination module is used to determine the user's brain fatigue index by extracting the energy components of the EEG signal in the Delta, Theta, Alpha, Beta, and Gamma bands; and to determine the user's ECG fatigue index by performing fast Fourier transform and power spectrum analysis on the ECG signal. The respiratory fatigue index of the user is calculated by collecting the peak extreme points in the respiratory electrical signal; The eye fatigue index of the user was determined based on the fixation duration and eye closure duration of blinking movements in the eye movement images. The fusion module is used to use EEG fatigue index, ECG fatigue index, respiratory fatigue index, and eye movement fatigue index as evidence to calculate the basic probability allocation (BPA) for each piece of evidence corresponding to different levels of user fatigue, resulting in EEG evidence BPA, ECG evidence BPA, respiratory evidence BPA, and eye movement evidence BPA. DS evidence theory is then used to perform decision-level fusion of the EEG evidence BPA, ECG evidence BPA, respiratory evidence BPA, and eye movement evidence BPA to calculate the user's comprehensive fatigue index. The fatigue detection module is used to detect the user's fatigue status based on the comprehensive fatigue index.

9. A computer device for fatigue state detection, characterized in that, include: A memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the fatigue state detection method according to any one of claims 1-7.

10. A readable storage medium, characterized in that, The readable storage medium stores a computer program, which includes program instructions that, when executed by a processor, perform the steps of the fatigue state detection method according to any one of claims 1-7.

Images