Methods, systems, and media for constructing facial fatigue datasets for train drivers

Abstract

This invention discloses a method, system, and medium for constructing a facial fatigue dataset suitable for train drivers, relating to the field of data processing. The method includes acquiring multi-dimensional data of a subject performing an AX-CPT psychological paradigm task, including the subject's facial data, heart rate data, reaction time of the AX-CPT task, and accuracy of the AX-CPT task. The subject's fatigue state is labeled using the heart rate data, reaction time, and accuracy of the AX-CPT task. A facial fatigue dataset is constructed based on the labeled facial data and corresponding fatigue state labels. By performing the AX-CPT psychological paradigm task, an accurate facial fatigue dataset suitable for the working characteristics of train drivers can be obtained, thus laying the foundation for accurate detection of fatigue states in actual working scenarios for train drivers.

Description

Technical Field

[0001] This invention relates to the field of data processing, and in particular to a method, system, and medium for constructing a facial fatigue dataset suitable for the working characteristics of train drivers. Background Technology

[0002] Driver fatigue is a significant human factor in train accidents, and effective driver fatigue risk management is crucial for accident prevention. Train drivers' duties are characterized by long working hours, high stress, and monotonous tasks, making driver fatigue almost inevitable. With the development of image recognition and deep learning technologies, high-quality datasets can significantly improve model training quality and prediction accuracy. Constructing a facial fatigue dataset suitable for the working characteristics of train drivers lays the foundation for the development of subsequent fatigue detection systems.

[0003] The construction of existing facial fatigue datasets requires capturing facial reactions when fatigued. Existing methods generally fall into two categories. One involves posing the subject to demonstrate fatigue, but this method doesn't genuinely induce fatigue and deviates from true fatigue levels. The other method induces fatigue by depriving the subject of sleep; however, this approach is prolonged and unsuitable for the work characteristics of train drivers. Furthermore, current assessments of fatigue are subjective, and relying on human judgment to determine fatigue levels is inaccurate. Summary of the Invention

[0004] The purpose of this invention is to provide a method, system, and medium for constructing a facial fatigue dataset suitable for train drivers, which can accurately obtain a facial fatigue dataset suitable for the working characteristics of train drivers, thereby laying the foundation for accurate detection of fatigue state in actual working scenarios of train drivers.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A method for constructing a facial fatigue dataset for train drivers, the method comprising:

[0007] Acquire multi-dimensional data of the subject performing the AX-CPT psychological paradigm task; the multi-dimensional data includes the subject's facial data, the subject's heart rate data, the reaction time of the AX-CPT task, and the accuracy of the AX-CPT task;

[0008] The fatigue state was calibrated using the heart rate data, reaction time of the AX-CPT task, and accuracy of the AX-CPT task of the subjects.

[0009] The facial fatigue dataset is constructed based on the facial data of the subjects labeled as being in a fatigued state and the corresponding fatigue state labels.

[0010] Optionally, obtain multi-dimensional data on the subjects performing the AX-CPT psychological paradigm task, specifically including:

[0011] Preparation and training phase: The subjects practice the basic operations of the AX-CPT psychological paradigm. Once the performance indicators meet the preset requirements, the formal experiment begins. The performance indicators include the subjects' reaction time and accuracy in the AX-CPT task.

[0012] Fatigue state detection and baseline performance acquisition phase: The subject completes a total of 10 minutes of AX-CPT task, and the subject's heart rate data, AX-CPT task reaction time, AX-CPT task accuracy and the subject's facial data are collected.

[0013] AX-CPT fatigue induction phase: The experiment lasted a total of 90 minutes and was divided into 9 consecutive test phases, each lasting 10 minutes. During each test phase of the AX-CPT task performed by the subject, the subject's heart rate data, reaction time of the AX-CPT task, accuracy of the AX-CPT task, and facial data were collected.

[0014] Optionally, before calibrating the fatigue state using the subject's heart rate data, the reaction time of the AX-CPT task, and the accuracy of the AX-CPT task, the method further includes: normalizing the subject's heart rate data, the reaction time of the AX-CPT task, and the accuracy of the AX-CPT task.

[0015] Optionally, the expression used to calibrate the fatigue state is:

[0016]

[0017] Y = 0.1(HR - HR) b )+0.15(RT-RT b )+0.2(ACC-ACC b )

[0018] Where Z < 0.5, the subject is in a non-fatigued state based on the current heart rate data, reaction time of the AX-CPT task, and accuracy of the AX-CPT task; when Z > 0.5, the subject is in a fatigued state based on the current heart rate data, reaction time of the AX-CPT task, and accuracy of the AX-CPT task. b RTb ACC b These represent the baseline data for heart rate (HR), reaction time (RT) of the AX-CPT task, and accuracy (ACC) of the AX-CPT task, respectively.

[0019] Optionally, after constructing the facial fatigue dataset based on the facial data of the subjects labeled as being in a fatigued state and the corresponding fatigue state labels, the method further includes: detecting the fatigue state of the train driver based on the facial fatigue dataset, specifically:

[0020] A deep learning network is trained using the aforementioned facial fatigue dataset; the trained deep learning network is then used to detect the fatigue state of the train driver.

[0021] This invention also provides a system for constructing a facial fatigue dataset for train drivers, the system comprising:

[0022] The data acquisition module is used to acquire multi-dimensional data of the subject performing the AX-CPT psychological paradigm task; the multi-dimensional data includes the subject's facial data, the subject's heart rate data, the reaction time of the AX-CPT task, and the accuracy of the AX-CPT task;

[0023] The state calibration module is used to calibrate the fatigue state using the subject's heart rate data, reaction time of the AX-CPT task, and accuracy of the AX-CPT task.

[0024] The dataset construction module is used to construct the facial fatigue dataset based on the facial data of the subject labeled as being in a fatigued state and the corresponding fatigue state labels.

[0025] The state detection module is used to detect the fatigue state of train drivers based on the facial fatigue dataset.

[0026] Optionally, the system further includes a normalization module; the normalization module is used to normalize the heart rate data of the subject, the reaction time of the AX-CPT task, and the accuracy of the AX-CPT task before calibrating the fatigue state using the heart rate data of the subject, the reaction time of the AX-CPT task, and the accuracy of the AX-CPT task.

[0027] Optionally, the expression used to calibrate the fatigue state is:

[0028]

[0029] Y = 0.1(HR - HR) b )+0.15(RT-RT b )+0.2(ACC-ACCb )

[0030] Where Z < 0.5, the subject is in a non-fatigued state based on the current heart rate data, reaction time of the AX-CPT task, and accuracy of the AX-CPT task; when Z > 0.5, the subject is in a fatigued state based on the current heart rate data, reaction time of the AX-CPT task, and accuracy of the AX-CPT task. b RT b ACC b These represent the baseline data for heart rate (HR), reaction time (RT) of the AX-CPT task, and accuracy (ACC) of the AX-CPT task, respectively.

[0031] Optionally, the system further includes a state detection module; the state detection module is used to detect the fatigue state of the train driver based on the facial fatigue dataset.

[0032] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method for constructing a facial fatigue dataset suitable for train drivers.

[0033] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0034] This invention provides a method, system, and medium for constructing a facial fatigue dataset suitable for train drivers. It induces fatigue in the subject by performing an AX-CPT psychological paradigm task. By acquiring the subject's heart rate, reaction time, accuracy, and facial data during the AX-CPT task, and calibrating the collected data with the fatigue state, a facial fatigue dataset under fatigued conditions is accurately obtained. This dataset is suitable for the working characteristics of train drivers and facilitates accurate detection of fatigue in train drivers. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a flowchart of a method for constructing a facial fatigue dataset for train drivers, provided in Embodiment 1 of the present invention.

[0037] Figure 2This is a flowchart of fatigue data acquisition provided in Embodiment 1 of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] The purpose of this invention is to provide a method, system, and medium for constructing a facial fatigue dataset suitable for train drivers, addressing the problems of driver fatigue induction and state calibration in train driver work scenarios. A fatigue induction process suitable for train driver work scenarios and a scheme for judging driver fatigue state through multi-dimensional data are established. This method has advantages such as stability, speed, and safety in fatigue induction, and in fatigue state calibration, the comprehensive judgment of fatigue state through multi-dimensional data makes fatigue judgment more accurate.

[0040] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0041] Example 1

[0042] like Figure 1 As shown in the figure, this embodiment provides a method for constructing a facial fatigue dataset suitable for train drivers, the method including:

[0043] S1: Obtain multi-dimensional data of the subject performing the AX-CPT psychological paradigm task; the multi-dimensional data includes the subject's facial data, the subject's heart rate data, the reaction time of the AX-CPT task, and the accuracy of the AX-CPT task.

[0044] In real-world train transportation scenarios, the impact of uncertain environments leads to prolonged and unstable fatigue among drivers. To achieve stable fatigue levels within a shorter timeframe, short-term and controllable fatigue induction methods are needed. Given the continuous alertness and low-control nature of train driver tasks, long-duration cognitive tasks share similarities with the work characteristics of train drivers. The main principle behind fatigue induction through long-duration cognitive tasks is that the subject needs to handle high-intensity cognitive tasks for an extended period, thereby inducing a state of fatigue.

[0045] The AX-CPT psychological paradigm was used to collect data on various dimensions of fatigue in the subjects being photographed. The photographing process should be conducted in a closed, quiet room. The entire fatigue induction process includes three stages, and the entire induction process is as follows: Figure 2 As shown.

[0046] a. The first stage is the preparation and training stage. Participants need to familiarize themselves with the basic operation of the AX-CPT paradigm. Only after meeting the performance indicators can they proceed to the formal experiment. These performance indicators include the participant's key press response time and judgment accuracy. The AX-CPT paradigm is a widely used paradigm for long-term cognitive response inhibition tasks induced by mental fatigue, designed to assess participants' attention and inhibition abilities when performing continuous response tasks. This task involves two different types of stimuli: a target stimulus and a non-target stimulus. Every four letters form a pair, with the middle two being distractors. Only combinations where the first letter is "A" and the last letter is "X" are target stimuli; all other combinations are non-target stimuli. Specifically, the induced paradigm includes the following four letter pair combinations: target stimulus "A-distractor letter-distractor letter-X"; non-target stimuli "non-A-two random distractors-X", "non-A-two random distractors-non-X", and "A-two random distractors-non-X". In the AX-CPT paradigm, subjects are required to respond sequentially to two types of stimuli in different pairs, with a 1000ms interval between each pair for the subject's response. Subjects need to learn to correctly predict the probability of the target stimulus appearing in different pairs and to respond quickly when the target stimulus appears. Conversely, when a non-target stimulus appears, subjects need to suppress their response.

[0047] b. The second phase was the fatigue state detection and baseline performance acquisition phase. During this phase, participants completed a total of 10 minutes of the AX-CPT task, and psychological data such as heart rate, reaction time, accuracy, and facial data were collected. Physiological data were collected using the MGY-H12 dynamic Holter medical-grade ambulatory electrocardiogram monitor from Beijing Megotech Software Technology Co., Ltd.

[0048] c. The third stage is the AX-CPT fatigue induction stage, with a total experimental duration of 90 minutes, divided into 9 test phases, each lasting approximately 10 minutes. The data collection process is similar to stage b, but needs to be conducted continuously for 90 minutes. During each test phase of the AX-CPT task performed by the subject, the subject's heart rate data, AX-CPT task reaction time, AX-CPT task accuracy, and facial data are collected.

[0049] S2: The fatigue state is calibrated using the heart rate data of the subject, the reaction time of the AX-CPT task, and the accuracy of the AX-CPT task.

[0050] This invention first normalizes the heart rate (HR), reaction time (RT), and accuracy (ACC) of the AX-CPT task obtained in step one above. The normalization formula is as follows:

[0051]

[0052] Based on the above normalized indices, the specific formula for fatigue state analysis is as follows:

[0053] Y = 0.1(HR - HR) b )+0.15(RT-RT b )+0.2(ACC-ACC b )

[0054] Among them, HR b RT b ACC b This represents baseline data for heart rate (HR), reaction time (RT) of the AX-CPT task, and accuracy (ACC) of the AX-CPT task. Since the onset of fatigue varies from person to person (individual variability), we need to subtract the baseline during later fatigue calibration to further reduce individual variability.

[0055] This invention uses logistic regression to determine fatigue state, wherein the logistic regression formula is:

[0056]

[0057] If Z < 0.5, the state is considered non-fatigue; if Z > 0.5, the state is considered fatigued.

[0058] S3: Construct the facial fatigue dataset based on the facial data of the subject labeled as being in a fatigued state and the corresponding fatigue state label.

[0059] Through the above S1 and S2 steps, a facial fatigue dataset suitable for the working characteristics of train drivers is constructed. This method stably and quickly induces the fatigue state of the subject, and the use of multi-dimensional data to determine the fatigue state of the subject makes the labeling of fatigue state more accurate.

[0060] The purpose of constructing a face fatigue dataset is to provide a dataset for the training of future deep learning networks. Therefore, after constructing the face fatigue dataset, the following steps S3 are further included.

[0061] S4: Detect the fatigue state of the train driver based on the aforementioned facial fatigue dataset. Specifically:

[0062] A deep learning network is trained using the aforementioned facial fatigue dataset; the trained deep learning network is then used to detect the fatigue state of the train driver.

[0063] In this embodiment, a facial fatigue dataset suitable for the working characteristics of train drivers is constructed. This method stably and quickly induces the fatigue state of the subject, and the use of multi-dimensional data to determine the fatigue state of the subject makes the labeling of fatigue state more accurate. Using the labeled facial fatigue dataset to detect the fatigue of train drivers in real-world scenarios is more accurate.

[0064] Example 2

[0065] This embodiment provides a system for constructing a facial fatigue dataset for train drivers, the system comprising:

[0066] The data acquisition module is used to acquire multi-dimensional data of the subject performing the AX-CPT psychological paradigm task; the multi-dimensional data includes the subject's facial data, the subject's heart rate data, the reaction time of the AX-CPT task, and the accuracy of the AX-CPT task.

[0067] The state calibration module is used to calibrate the fatigue state using the subject's heart rate data, reaction time of the AX-CPT task, and accuracy of the AX-CPT task.

[0068] The expression used to calibrate the fatigue state is as follows:

[0069]

[0070] Y = 0.1(HR - HR) b )+0.15(RT-RT b )+0.2(ACC-ACC b )

[0071] Where Z < 0.5, the subject is in a non-fatigued state based on the current heart rate data, reaction time of the AX-CPT task, and accuracy of the AX-CPT task; when Z > 0.5, the subject is in a fatigued state based on the current heart rate data, reaction time of the AX-CPT task, and accuracy of the AX-CPT task. b RT b ACC b These represent the baseline data for heart rate (HR), reaction time (RT) of the AX-CPT task, and accuracy (ACC) of the AX-CPT task, respectively.

[0072] The dataset construction module is used to construct the facial fatigue dataset based on the facial data of the subject labeled as being in a fatigued state and the corresponding fatigue state labels.

[0073] The state detection module is used to detect the fatigue state of train drivers based on the facial fatigue dataset.

[0074] The system further includes a normalization module; the normalization module is used to normalize the heart rate data of the subject, the reaction time of the AX-CPT task, and the accuracy of the AX-CPT task before calibrating the fatigue state using the heart rate data of the subject, the reaction time of the AX-CPT task, and the accuracy of the AX-CPT task.

[0075] Example 3

[0076] This embodiment provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor runs the computer program to enable the electronic device to execute the method for constructing a facial fatigue dataset for train drivers as described in Embodiment 1.

[0077] Alternatively, the aforementioned electronic device may be a server.

[0078] In addition, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method for constructing a facial fatigue dataset for train drivers as described in Embodiment 1.

[0079] Embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0080] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0081] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0082] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0083] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0084] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for constructing a facial fatigue dataset suitable for train drivers, characterized in that, The method includes: Acquire multi-dimensional data of the subject performing the AX-CPT psychological paradigm task; the multi-dimensional data includes the subject's facial data, the subject's heart rate data, the reaction time of the AX-CPT task, and the accuracy of the AX-CPT task; The fatigue state was calibrated using the heart rate data, reaction time of the AX-CPT task, and accuracy of the AX-CPT task of the subjects. The facial fatigue dataset is constructed based on the facial data of the subjects labeled as being in a fatigued state and the corresponding fatigue state labels.

2. The method according to claim 1, characterized in that, Obtain multi-dimensional data on the subjects performing AX-CPT psychological paradigm tasks, specifically including: Preparation and training phase: The subjects practice the basic operations of the AX-CPT psychological paradigm. Once the performance indicators meet the preset requirements, the formal experiment begins. The performance indicators include the subjects' reaction time and accuracy in the AX-CPT task. Fatigue state detection and baseline performance acquisition phase: The subject completes a total of 10 minutes of AX-CPT task, and the subject's heart rate data, AX-CPT task reaction time, AX-CPT task accuracy and the subject's facial data are collected. AX-CPT fatigue induction phase: The experiment lasted a total of 90 minutes and was divided into 9 consecutive test phases, each lasting 10 minutes. During each test phase of the AX-CPT task performed by the subject, the subject's heart rate data, reaction time of the AX-CPT task, accuracy of the AX-CPT task, and facial data were collected.

3. The method according to claim 1, characterized in that, Before calibrating the fatigue state using the subject's heart rate data, reaction time of the AX-CPT task, and accuracy of the AX-CPT task, the method further includes: normalizing the subject's heart rate data, reaction time of the AX-CPT task, and accuracy of the AX-CPT task.

4. The method according to claim 3, characterized in that, The expression used to calibrate fatigue state is: Y=0.1(HR-HR b )+0.15(RT-RT b )+0.2(ACC-ACC b ) Where Z < 0.5, the subject is in a non-fatigued state based on the current heart rate data, reaction time of the AX-CPT task, and accuracy of the AX-CPT task; when Z > 0.5, the subject is in a fatigued state based on the current heart rate data, reaction time of the AX-CPT task, and accuracy of the AX-CPT task. b RT b ACC b These represent the baseline data for heart rate (HR), reaction time (RT) of the AX-CPT task, and accuracy (ACC) of the AX-CPT task, respectively.

5. The method according to claim 1, characterized in that, After constructing the facial fatigue dataset based on the facial data of the subjects labeled as fatigued and the corresponding fatigue state labels, the method further includes: detecting the fatigue state of the train driver based on the facial fatigue dataset, specifically: A deep learning network is trained using the aforementioned facial fatigue dataset; the trained deep learning network is then used to detect the fatigue state of the train driver.

6. A system for constructing a facial fatigue dataset for train drivers, characterized in that, The system includes: The data acquisition module is used to acquire multi-dimensional data of the subject performing the AX-CPT psychological paradigm task; the multi-dimensional data includes the subject's facial data, the subject's heart rate data, the reaction time of the AX-CPT task, and the accuracy of the AX-CPT task; The state calibration module is used to calibrate the fatigue state using the subject's heart rate data, reaction time of the AX-CPT task, and accuracy of the AX-CPT task. The dataset construction module is used to construct the facial fatigue dataset based on the facial data of the subject labeled as being in a fatigued state and the corresponding fatigue state labels.

7. The system according to claim 6, characterized in that, The system also includes a normalization module; the normalization module is used to normalize the heart rate data of the subject, the reaction time of the AX-CPT task, and the accuracy of the AX-CPT task before calibrating the fatigue state using the heart rate data of the subject, the reaction time of the AX-CPT task, and the accuracy of the AX-CPT task.

8. The system according to claim 6, characterized in that, The expression used to calibrate fatigue state is: Y=0.1(HR-HR b )+0.15(RT-RT b )+0.2(ACC-ACC b ) Where Z < 0.5, the subject is in a non-fatigued state based on the current heart rate data, reaction time of the AX-CPT task, and accuracy of the AX-CPT task; when Z > 0.5, the subject is in a fatigued state based on the current heart rate data, reaction time of the AX-CPT task, and accuracy of the AX-CPT task. b RT b ACC b These represent the baseline data for heart rate (HR), reaction time (RT) of the AX-CPT task, and accuracy (ACC) of the AX-CPT task, respectively.

9. The system according to claim 6, characterized in that, The system also includes a state detection module; the state detection module is used to detect the fatigue state of train drivers based on the facial fatigue dataset.

10. A computer-readable storage medium storing a computer program, characterized in that, When executed by a processor, the computer program implements the method for constructing a facial fatigue dataset for train drivers as described in any one of claims 1 to 5.

Images