Method and device for monitoring fatigue driving of railway driver and electronic equipment

Abstract

The invention provides a railway driver fatigue driving monitoring method and device and electronic equipment, and relates to the field of data processing. Physiological data, behavior characteristic data and driving operation data of the railway driver in the driving process are obtained, the physiological data comprise heart rate variability, brain wave characteristics and electrodermal response, and the behavior characteristic data comprise blinking frequency, eye closing time, fixation point change and yawning times; the driving operation data comprises a speed regulation handle rotation angle, acceleration change and braking frequency; inputting the physiological data, the behavior characteristic data and the driving operation data into an LSTM model to obtain a physiological fatigue score, a behavior characteristic fatigue score and a driving operation fatigue score; determining a comprehensive fatigue index based on the physiological fatigue score, the behavior characteristic fatigue score and the driving operation fatigue score; and judging whether the railway driver is in fatigue driving or not in the driving process according to the comprehensive fatigue index. By implementing the technical scheme, the monitoring accuracy is improved.

Description

Technical Field

[0001] The present application relates to the technical field of data processing, and in particular to a method, device and electronic equipment for monitoring fatigue driving of railway drivers. Background Art

[0002] In the railway transportation industry, the fatigue state of railway drivers directly affects the safety of train operation. Fatigue driving not only reduces the driver's reaction speed and concentration, but may also lead to major accidents.

[0003] At present, traditional fatigue monitoring methods are mostly based on linear regression or simple statistical models, which make it difficult to capture the nonlinear and temporal relationships between different features. Moreover, these models cannot fully utilize the potential of multidimensional data, resulting in low monitoring accuracy.

[0004] Therefore, a method, device and electronic equipment for monitoring fatigue driving of railway drivers are urgently needed. Summary of the invention

[0005] The present application provides a method, device and electronic equipment for monitoring railway driver fatigue driving, which is convenient for improving monitoring accuracy.

[0006] In a first aspect of the present application, a method for monitoring fatigue driving of a railway driver is provided, the method comprising: obtaining physiological data, behavioral characteristic data and driving operation data of the railway driver during driving, the physiological data comprising heart rate variability, brain wave characteristics and skin electrical response, the behavioral characteristic data comprising blinking frequency, eye closing time, gaze point change and yawning frequency, and the driving operation data comprising speed control handle rotation angle, acceleration change and braking frequency; inputting the physiological data, the behavioral characteristic data and the driving operation data into an LSTM model to obtain a physiological fatigue score, a behavioral characteristic fatigue score and a driving operation fatigue score; determining a comprehensive fatigue index based on the physiological fatigue score, the behavioral characteristic fatigue score and the driving operation fatigue score; and judging whether the railway driver is driving fatigued during driving according to the comprehensive fatigue index.

[0007] By adopting the above technical solution, by obtaining the physiological data, behavioral characteristic data and driving operation data of railway drivers, this method can comprehensively monitor the driver's fatigue status from multiple angles. This multi-dimensional data fusion method can more comprehensively reflect the driver's fatigue level and make up for the monitoring blind spots that may be caused by a single data source. By inputting heart rate variability, brain wave characteristics, skin galvanic response, blinking frequency, eye closure time, gaze point change, yawning number, speed control handle rotation angle, acceleration change, and braking frequency into the LSTM model respectively, more accurate physiological fatigue scores, behavioral characteristic fatigue scores, and driving operation fatigue scores can be obtained. This hierarchical fatigue scoring method is helpful to analyze the contribution of different factors to fatigue. Using the LSTM model to predict fatigue status can effectively capture the complex nonlinear relationships and time series characteristics in time series data. The advantage of LSTM for time series data enables it to more accurately understand the dynamic changes of driver fatigue, which is more flexible and adaptable than traditional linear models. By integrating physiological fatigue scores, behavioral characteristic fatigue scores, and driving operation fatigue scores to form a comprehensive fatigue index, the driver's fatigue status can be more comprehensively evaluated. Based on the comprehensive fatigue index, the fatigue state of the railway driver during driving can be judged in real time, so as to achieve timely intervention. This real-time monitoring capability helps to identify the risk of fatigue driving in advance, reduce the probability of potential accidents, ensure the safety of train operation, and further improve the accuracy of monitoring railway driver fatigue driving.

[0008] Optionally, the obtaining of the physiological data, behavioral characteristic data and driving operation data of the railway driver during the driving process specifically includes: receiving original physiological data sent by a smart wearable device, the railway driver wears the smart wearable device, and the smart wearable device includes a smart watch and an EEG headband; receiving cab image data sent by a cab monitoring device; performing facial image processing on the cab image data using an image processing algorithm to obtain original behavioral characteristic data; receiving CAN bus data sent by an on-board sensor device, the on-board sensor device includes a speed control handle sensor device, an acceleration sensor device and a brake pressure sensor device; classifying and processing the CAN bus data to obtain original driving operation data; and obtaining the physiological data, the behavioral characteristic data and the driving operation data by preprocessing the original physiological data, the original behavioral characteristic data and the original driving operation data.

[0009] By adopting the above technical solutions, the physiological data of railway drivers can be collected in real time by using smart wearable devices. These devices can easily monitor physiological indicators such as heart rate and brain waves without the need for complex traditional medical equipment, which enhances the convenience and flexibility of data collection. At the same time, smart wearable devices usually have high wearing comfort and can monitor for a long time, reducing interference with the driver's work. By integrating data from different sources, this method can provide a comprehensive assessment of the driver's status. Multi-source data fusion helps to understand the causes of fatigue in a wider range and overcome the limitations of a single data source. By receiving image data sent by the cab monitoring device and combining it with image processing algorithms, the driver's facial features can be extracted. This visual monitoring can intuitively reflect the driver's status and help identify potential signs of fatigue. The application of image processing algorithms improves the degree of automation of data collection and reduces the need for manual intervention. Receiving CAN bus data sent by on-board sensor devices can monitor driving operation data in real time. These data are crucial for judging the driver's driving behavior and driving habits. By analyzing driving operation data, fatigue characteristics during driving can be revealed to help judge the driver's fatigue level. By preprocessing the raw physiological data, behavioral characteristic data, and driving operation data, noise and outliers can be eliminated, and the accuracy and reliability of the data can be improved. The structured data acquisition process of this method supports efficient real-time monitoring and can continuously track the physiological and behavioral status of the driver during driving. This real-time nature enables the monitoring system to respond to changes in driver fatigue in a timely manner, implement necessary safety measures, and ensure train safety.

[0010] Optionally, the physiological data, the behavioral characteristic data and the driving operation data are all input into the LSTM model to obtain a physiological fatigue score, a behavioral characteristic fatigue score and a driving operation fatigue score, specifically using the following calculation formula: ; in, is the physiological fatigue score, is the individual weight of the physiological data, is a nonlinear activation function, is the value of the ith physiological characteristic, is the time attenuation factor of the physiological characteristic, t is the time, For physiological characteristics and physiological characteristics The interaction weight between is the value of the jth physiological characteristic; ; in, is the behavioral characteristic fatigue score, is the individual weight of the behavioral feature data, is a nonlinear activation function, is the value of the kth behavior feature, is the time decay factor of the behavior characteristic, t is the time, Behavioral characteristics and behavioral characteristics The interaction weight between is the value of the lth behavior feature; ; in, is the driving operation fatigue score, is the individual weight of the driving operation data, is a nonlinear activation function, is the value of the pth driving operation feature, is the time attenuation factor of driving operation characteristics, t is time, Driving operation characteristics and driving operation characteristics The interaction weight between is the value of the qth driving operation feature.

[0011] By adopting the above technical solutions, the individual weights in the formula assign different influences to each feature, allowing the model to perform personalized assessments based on the individual differences of each driver. Different drivers may have different fatigue responses to the same feature, and this personalized processing can improve the accuracy of fatigue assessment. The use of nonlinear activation functions allows the model to capture complex nonlinear relationships. The changes in fatigue status are often nonlinear. Through nonlinear activation functions, the model can more effectively learn the complex interactions between physiological, behavioral, and driving characteristics, improving the performance of the model. The time decay factor in the formula takes into account the changes in the importance of different features over time. Over time, the impact of some features on fatigue status may weaken, while other features may become more important. This dynamic adjustment mechanism enables the model to better reflect the real-time fatigue status and provide more accurate fatigue assessments. By considering the interaction weights between physiological features, behavioral features, and driving operation features, the model can identify the joint impact of multiple feature combinations on fatigue status. This consideration of feature interactions can more comprehensively reflect complex fatigue mechanisms and avoid information loss caused by relying on a single feature.

[0012] Optionally, the comprehensive fatigue index is determined based on the physiological fatigue score, the behavioral characteristic fatigue score and the driving operation fatigue score, specifically using the following calculation formula: ; in, is the comprehensive fatigue index, is the overall weight of physiological data, is the overall weight of the behavioral feature data, is the overall weight of the driving operation data.

[0013] By adopting the above technical solutions, the comprehensive fatigue index brings together data from three dimensions: physiology, behavior, and driving operation, so that it can fully reflect the driver's fatigue status. Through comprehensive evaluation, the driver's fatigue level can be captured more accurately, avoiding misjudgments that may be caused by single-dimensional evaluation. The overall weight in the formula can be adjusted according to different application scenarios and specific needs. This flexibility allows the comprehensive fatigue index to adapt to different environments and individual differences, providing a more personalized fatigue assessment. Combining multiple fatigue scores into a single fatigue index can simplify the judgment and decision-making process of fatigue status. Managers and operators can quickly evaluate the driver's fatigue status based on this comprehensive index, thereby improving response efficiency. The calculation of the comprehensive fatigue index supports real-time monitoring, allowing the management system to quickly obtain the driver's fatigue status and take timely intervention measures. This real-time feedback mechanism helps to avoid potential safety risks at critical moments.

[0014] Optionally, judging whether the railway driver is driving in fatigue during driving based on the comprehensive fatigue index specifically includes: obtaining historical driving data of the railway driver during historical driving, the historical driving data including marked fatigue driving status and corresponding historical physiological data, historical behavioral characteristic data and historical driving operation data; performing statistical analysis on the historical driving data using a ROC curve to obtain a fatigue index threshold; comparing the comprehensive fatigue index with the fatigue index threshold; if it is determined that the comprehensive fatigue index is greater than or equal to the fatigue index threshold, determining that the railway driver is driving in fatigue; if it is determined that the comprehensive fatigue index is less than the fatigue index threshold, determining that the railway driver is not driving in fatigue.

[0015] By adopting the above technical solution, by obtaining the historical driving data of railway drivers in the historical driving process, including the marked fatigue driving state and the corresponding physiological, behavioral characteristics and driving operation data, the method can establish a scientific basis for judgment. This analysis based on real data helps to improve the accuracy and reliability of judgment. The use of ROC curve to perform statistical analysis on historical driving data helps to determine the optimal fatigue index threshold. The ROC curve can effectively evaluate the performance of the classification model, help identify the true positive rate and false positive rate under different thresholds, so as to select the threshold that can maximize sensitivity and specificity and improve the recognition ability of fatigue state. The fatigue index threshold obtained by historical data analysis can be dynamically adjusted with time and conditions, making the fatigue monitoring system more flexible and adaptable. This dynamic adaptability can better cope with the fatigue state of different environments and different drivers, and avoid misjudgment caused by fixed thresholds. Comparing the comprehensive fatigue index with the fatigue index threshold can quickly and accurately judge the driver's fatigue state. By setting the threshold, the risk of misjudgment of fatigue judgment can be effectively reduced. In the identification of fatigue state, using ROC curve analysis to optimize the threshold can improve the recognition rate of fatigue driving and reduce safety hazards caused by misjudgment.

[0016] Optionally, the method also includes: if it is determined that the comprehensive fatigue index is greater than or equal to the fatigue index threshold, triggering a first-level alarm mechanism, and the first-level alarm mechanism is used to prompt the railway driver that he is driving fatigued; if it is determined that the comprehensive fatigue index is greater than or equal to the fatigue index threshold, and is greater than or equal to the corresponding preset duration or exceeds a preset number of times, triggering a second-level alarm mechanism, and the second-level alarm mechanism is used to prompt the railway driver to take a rest and change drivers.

[0017] By adopting the above technical solution, the first-level alarm mechanism will immediately issue an alarm when the driver's comprehensive fatigue index reaches the threshold, which can timely remind the driver to pay attention to his own fatigue state. This real-time feedback mechanism can help drivers quickly realize potential fatigue risks and take appropriate actions. By setting up a second-level alarm mechanism, this method can achieve hierarchical management of fatigue monitoring. When the comprehensive fatigue index not only exceeds the threshold, but also reaches a specific duration or exceeds the preset number of times, a more serious alarm will be triggered. This hierarchical mechanism can ensure that more stringent safety measures are taken to ensure driving safety when the fatigue level increases. Through timely alarms, safety accidents caused by fatigue driving can be effectively prevented. When the driver realizes that he is in a fatigued state, he can take the initiative to choose to rest or change the driver to avoid traffic accidents caused by slow reaction or misjudgment. When the second-level alarm mechanism is triggered, the driver is prompted to rest and change the driver, which can achieve scientific fatigue management. This management strategy not only takes into account the driver's fatigue state, but also helps maintain overall driving safety and ensure the safe operation of the train. When the driver takes a rest in a fatigued state, he can maintain better work efficiency and concentration. Scientific and reasonable rest arrangements help improve the driver's work performance, thereby improving overall operational efficiency.

[0018] Optionally, before the physiological data, the behavioral characteristic data and the driving operation data are input into the LSTM model to obtain the physiological fatigue score, the behavioral characteristic fatigue score and the driving operation fatigue score, the LSTM model is trained; the training of the LSTM model specifically includes: obtaining training data; performing missing value processing, data standardization and data slicing on the training data to obtain time series data; performing feature engineering on the time series data to obtain an initial model; determining a training set and a validation set based on the training data; and optimizing the initial model using the training set and the validation set to obtain the LSTM model.

[0019] By adopting the above technical solutions, by obtaining real training data, we can ensure that the basis of model training is reliable and representative. An effective training data set can improve the generalization ability and prediction accuracy of the model. Real data can reflect the fatigue state in actual operation, help the model learn the relationship between different features and fatigue, and thus improve the prediction ability. By reasonably filling or deleting missing values, data integrity can be improved to ensure that the model will not be biased due to incomplete data during training, thereby improving the robustness of the model. Data standardization can eliminate the dimensional differences between different features so that all features are in the same range. This is particularly important for LSTM models because neural networks are sensitive to the scale of input data. Standardization helps to speed up model convergence, improve training efficiency, and reduce learning difficulties caused by feature scale differences, ultimately improving model performance. Data slicing is to divide continuous time series data into small segments so that the LSTM model can effectively learn the time series relationship. This process is particularly important for processing time series data. The sliced ​​data can provide clearer contextual information, allowing the LSTM model to capture the key time dependencies and feature changes in the time series and improve the accuracy of prediction. Feature engineering is a key step to improve model performance. The performance of the model can be significantly improved by selecting, transforming, and creating new features. Through feature engineering, features closely related to fatigue status can be extracted and unnecessary information can be reduced, so that the LSTM model can focus on important signals and improve prediction accuracy. The division of training and validation sets can prevent the model from overfitting and ensure its good performance on new data, thereby improving the credibility of the model. By optimizing the initial model, the hyperparameters and model structure can be further adjusted to find the optimal model configuration. The trained and optimized LSTM model can effectively process complex time series data, learn nonlinear relationships, and accurately predict fatigue status.

[0020] In a second aspect of the present application, a monitoring device for fatigue driving of a railway driver is provided, characterized in that the monitoring device includes an acquisition module and a processing module, wherein the acquisition module is used to acquire physiological data, behavioral characteristic data and driving operation data of the railway driver during driving, the physiological data includes heart rate variability, brain wave characteristics and skin electrical response, the behavioral characteristic data includes blinking frequency, eye closing time, gaze point change and yawning number, and the driving operation data includes speed control handle rotation angle, acceleration change and braking frequency; the processing module is used to input the physiological data, the behavioral characteristic data and the driving operation data into an LSTM model to obtain a physiological fatigue score, a behavioral characteristic fatigue score and a driving operation fatigue score; the processing module is also used to determine a comprehensive fatigue index based on the physiological fatigue score, the behavioral characteristic fatigue score and the driving operation fatigue score; the processing module is also used to judge whether the railway driver is driving fatigued during driving according to the comprehensive fatigue index.

[0021] In the third aspect of the present application, an electronic device is provided, which includes a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device performs the method described above.

[0022] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed, the method described above is executed.

[0023] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: By obtaining the physiological data, behavioral characteristic data and driving operation data of railway drivers, this method can comprehensively monitor the driver's fatigue status from multiple angles. This multi-dimensional data fusion method can more comprehensively reflect the driver's fatigue level and make up for the monitoring blind spots that may be caused by a single data source. By inputting heart rate variability, brain wave characteristics, skin galvanic response, blinking frequency, eye closure time, gaze point change, yawning number, speed control handle rotation angle, acceleration change, and braking frequency into the LSTM model respectively, more accurate physiological fatigue scores, behavioral characteristic fatigue scores, and driving operation fatigue scores can be obtained. This hierarchical fatigue scoring method is helpful to analyze the contribution of different factors to fatigue. Using the LSTM model to predict fatigue status can effectively capture the complex nonlinear relationships and time series characteristics in time series data. The advantage of LSTM for time series data enables it to more accurately understand the dynamic changes of driver fatigue, which is more flexible and adaptable than traditional linear models. By integrating physiological fatigue scores, behavioral characteristic fatigue scores, and driving operation fatigue scores to form a comprehensive fatigue index, the driver's fatigue status can be more comprehensively evaluated. Based on the comprehensive fatigue index, the fatigue state of the railway driver during driving can be judged in real time, so as to achieve timely intervention. This real-time monitoring capability helps to identify the risk of fatigue driving in advance, reduce the probability of potential accidents, ensure the safety of train operation, and further improve the accuracy of monitoring railway driver fatigue driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A flow chart of a method for monitoring railway driver fatigue driving provided in an embodiment of the present application.

[0025] Figure 2 Another flowchart of a method for monitoring railway driver fatigue driving provided in an embodiment of the present application.

[0026] Figure 3 A schematic diagram of a module of a device for monitoring railway driver fatigue driving provided in an embodiment of the present application.

[0027] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0028] Explanation of the reference numerals: 31, acquisition module; 32, processing module; 41, processor; 42, communication bus; 43, user interface; 44, network interface; 45, memory. DETAILED DESCRIPTION

[0029] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0030] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0031] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0032] In the railway transportation industry, the fatigue state of railway drivers is crucial to the safe operation of trains. Fatigue driving not only significantly reduces the driver's reaction speed and concentration, but may also lead to serious traffic accidents, causing immeasurable losses to passengers and society.

[0033] At present, traditional fatigue monitoring methods usually rely on linear regression or simple statistical models. However, these traditional methods have significant limitations and are difficult to effectively capture the complex nonlinear relationships and temporal changes between different features. In addition, existing models often fail to fully utilize the potential of data from multiple dimensions, which leads to low accuracy of fatigue monitoring and is difficult to meet the needs of railway transportation safety management.

[0034] In order to solve the above technical problems, the present application provides a method for monitoring railway driver fatigue driving, referring to Figure 1 , Figure 1 A flow chart of a method for monitoring fatigue driving of a railway driver provided in an embodiment of the present application. The monitoring method is applied to a server and includes steps S110 to S140, which are as follows: S110. Acquire the physiological data, behavioral characteristic data and driving operation data of the railway driver during the driving process. The physiological data include heart rate variability, brain wave characteristics and skin electrical response. The behavioral characteristic data include blinking frequency, eye closing time, gaze point change and yawning frequency. The driving operation data include speed control handle rotation angle, acceleration change and braking frequency.

[0035] Specifically, the server is a computer system or cluster, which is a key component in the railway driver fatigue monitoring system and is responsible for receiving, processing, analyzing and feedback of data. Through its powerful computing and storage capabilities, the server can effectively monitor the driver's status and identify fatigue driving in a timely manner, thereby improving the safety of railway transportation. Physiological data reflects the driver's physical state and can help assess his fatigue level. These data are collected through smart wearable devices or physiological monitoring devices. Heart rate variability is an indicator of changes in the heart beat interval, which is used to reflect the activity of the autonomic nervous system. Lower heart rate variability is associated with fatigue, stress and health problems. If a driver's heart rate variability index decreases significantly over a period of time, it may indicate that he is in a state of fatigue and needs attention. Brain wave characteristics The brain wave data obtained through EEG equipment can reflect the driver's attention and fatigue status. Brain waves of specific frequencies, such as alpha waves and theta waves, are associated with relaxation and fatigue. Galvanic skin response is an indicator that reflects emotional and physiological states by measuring skin conductance, which is related to stress and excitement. If the galvanic skin response reading increases during driving, it may indicate that the driver feels stressed or fatigued, which in turn affects driving performance.

[0036] Secondly, behavioral characteristic data mainly reflects the behavioral patterns exhibited by drivers during driving, which can be used to assess their attention and fatigue. Blinking frequency is an important indicator of eye fatigue and attention. Frequent blinking or shorter blink intervals may indicate fatigue. If the driver's blinking frequency is monitored to increase by 50% within an hour, this may indicate that he is feeling tired. Eye closure time refers to the duration of the driver's eyes closed. Longer eye closure time may affect driving safety. If the driver closes his eyes for more than 5 seconds while driving, this will be a warning sign and may indicate fatigue. Monitoring the driver's gaze point can reveal his concentration level. If the gaze point changes frequently, it may mean that the driver is trying to stay awake. If the gaze point changes frequently in a short period of time, it may indicate that the driver is trying to distract himself, reflecting his fatigue state. Yawning is considered a clear sign of fatigue, and monitoring its frequency can help identify fatigue. If the driver yawns more than five times within a period of time, this can be regarded as a sign of fatigue. Finally, driving operation data reflects the driver's driving behavior and can help analyze his driving style and safety. The speed control handle rotation angle is used to monitor the driver's control ability over the speed control handle. Frequent and large turns may indicate a lack of concentration. Acceleration changes reflect the driver's driving performance during acceleration and deceleration. Smooth acceleration and deceleration means better driving control. Braking frequency reflects the driver's reaction ability and safe driving conditions. Frequent braking may indicate a decrease in sensitivity to the environment. If the driver is monitored to brake suddenly many times in a short period of time, this may be caused by a decrease in his alertness to the driving environment.

[0037] In a possible implementation, the physiological data, behavioral characteristic data and driving operation data of a railway driver during driving are obtained, specifically including: receiving original physiological data sent by a smart wearable device, the railway driver wears a smart wearable device, and the smart wearable device includes a smart watch and an EEG headband; receiving cab image data sent by a cab monitoring device; performing facial image processing on the cab image data using an image processing algorithm to obtain original behavioral characteristic data; receiving CAN bus data sent by an on-board sensor device, the on-board sensor device includes a speed control handle sensor device, an acceleration sensor device and a brake pressure sensor device; classifying and processing the CAN bus data to obtain original driving operation data; and obtaining physiological data, behavioral characteristic data and driving operation data by preprocessing the original physiological data, original behavioral characteristic data and original driving operation data.

[0038] Specifically, the physiological signals of railway drivers are monitored and collected through smart devices, which can collect data related to the driver's health status in real time. Smart watches monitor data such as heart rate, heart rate variability and skin galvanic response. If the driver feels tired while driving, the smart watch may record a sharp change in his heart rate or a decrease in skin galvanic response, indicating that there may be a risk of fatigue. EEG headbands are used to monitor brain wave activity and analyze the driver's attention level and fatigue status. When the driver drives for a long time, the EEG headband may monitor an increase in theta waves, which is related to fatigue and inattention. The cab monitoring device captures images in the cab in real time through a camera to observe the driver's behavior and status. The camera captures images of the driver yawning or closing his eyes while driving, which may indicate that the driver is experiencing fatigue. The acquired images are analyzed using image processing technology to extract data related to behavioral characteristics. Through facial expression recognition algorithms, the system can identify the number of yawns and blinking frequencies of the driver. These characteristics can be used to assess his fatigue level. CAN bus data can monitor driving operations in real time through on-board sensors such as speed control handle sensors, acceleration sensors and brake pressure sensors. The driver's driving stability can be judged by monitoring the rotation angle of the speed control handle. If the speed control handle rotates a large range, it may indicate that the driver's control ability is reduced when facing fatigue. The acceleration and deceleration behavior of the vehicle is monitored by the acceleration sensor device. If the acceleration changes too drastically, it may mean that the driver's reaction is not sensitive enough and further attention should be paid to his fatigue state. The braking frequency is recorded by the brake pressure sensor device to evaluate the driver's reaction ability. If the braking frequency is abnormally frequent, it may reflect the driver's driving behavior under the condition of decreased attention or fatigue. The collected CAN bus data is analyzed and classified to extract relevant driving operation data. Finally, the collected raw physiological data, behavioral characteristic data and driving operation data are sorted and cleaned to improve data quality and ensure the effectiveness of subsequent analysis. When processing raw data, preprocessing includes denoising, filtering and normalization. Missing value filling, standardization and feature selection can also be performed to ensure the consistency and reliability of the final data set.

[0039] S120, inputting the physiological data, the behavioral characteristic data and the driving operation data into the LSTM model to obtain a physiological fatigue score, a behavioral characteristic fatigue score and a driving operation fatigue score.

[0040] Specifically, LSTM is a variant of recurrent neural network, which is specially designed for processing and predicting time series data. LSTM can effectively capture the temporal relationship in the data and remember long-term and short-term dependencies, making it very suitable for monitoring fatigue status. By integrating the collected physiological data, behavioral characteristic data and driving operation data into a multi-dimensional input matrix. Each set of data is input into the LSTM model as a time step, so that it can learn the correlation between different time points. By inputting physiological data, behavioral characteristic data and driving operation data into the LSTM model, a multi-dimensional evaluation score for the driver's fatigue status can be generated. These fatigue scores will provide important basis for subsequent decision-making, such as prompting the driver to rest or take other safety measures, thereby improving the safety of railway transportation.

[0041] In a possible implementation, the physiological data, behavioral characteristic data, and driving operation data are all input into the LSTM model to obtain the physiological fatigue score, the behavioral characteristic fatigue score, and the driving operation fatigue score, specifically using the following calculation formula: ; in, is the physiological fatigue score, is the individual weight of the physiological data, is a nonlinear activation function, is the value of the ith physiological characteristic, is the time attenuation factor of the physiological characteristic, t is the time, For physiological characteristics and physiological characteristics The interaction weight between is the value of the jth physiological characteristic; ; in, is the behavioral characteristic fatigue score, is the individual weight of the behavioral feature data, is a nonlinear activation function, is the value of the kth behavior feature, is the time decay factor of the behavior characteristic, t is the time, Behavioral characteristics and behavioral characteristics The interaction weight between is the value of the lth behavior feature; ; in, is the driving operation fatigue score, is the individual weight of the driving operation data, is a nonlinear activation function, is the value of the pth driving operation feature, is the time attenuation factor of driving operation characteristics, t is time, Driving operation characteristics and driving operation characteristics The interaction weight between is the value of the qth driving operation feature.

[0042] Specifically, in the above formula, N is the number of physiological features, M is the number of behavioral features, and K is the number of driving operation features. The physiological fatigue score calculates the weighted sum of all physiological features, while taking into account the interaction effects between features. The introduction of nonlinear activation function and time decay factor enables the model to better reflect the impact of physiological signals on fatigue state. The behavioral feature fatigue score reflects the impact of behavioral features on fatigue state through weighted sum and feature interaction terms. The model can adapt to the behavioral changes of drivers during driving through nonlinear activation and time decay. The driving operation fatigue score calculates the weighted sum of all driving features, taking into account the interaction between features. This part can accurately reflect the impact of operating behavior on fatigue and help identify unstable driving operations. By combining the fatigue scores of physiological, behavioral and driving operation features, the comprehensive fatigue index can accurately reflect the fatigue state of the driver and can provide support for real-time monitoring and management. The design of each parameter and formula is to maximize the capture of the fatigue that may occur in the driver during driving, thereby improving safety and reducing the risk of accidents. In an embodiment of the present application, the nonlinear activation function can use a Sigmoid function.

[0043] S130: Determine a comprehensive fatigue index based on the physiological fatigue score, the behavioral characteristic fatigue score, and the driving operation fatigue score.

[0044] Specifically, the comprehensive fatigue index is formed by weighting the physiological fatigue score, behavioral characteristic fatigue score and driving operation fatigue score to form a comprehensive indicator, which aims to more comprehensively reflect the driver's overall fatigue status. By integrating multiple dimensions of physiological, behavioral and driving operation data, the calculated comprehensive fatigue index provides a more comprehensive and accurate method to monitor the fatigue status of railway drivers. This method not only helps to identify fatigue in a timely manner, but also effectively improves the safety of railway transportation and prevents potential accidents caused by fatigue driving.

[0045] In a possible implementation, a comprehensive fatigue index is determined based on the physiological fatigue score, the behavioral characteristic fatigue score, and the driving operation fatigue score, specifically using the following calculation formula: ; in, is the comprehensive fatigue index, is the overall weight of physiological data, is the overall weight of the behavioral feature data, is the overall weight of the driving operation data.

[0046] Specifically, suppose the following fatigue scores are collected for a railway driver: a physiological fatigue score of 70, representing high physiological fatigue, based on HRV and EEG data; a behavioral characteristic fatigue score of 60, based on the observed blinking frequency and yawning frequency; and a driving operation fatigue score of 50, based on driving operation data, such as the rotation angle of the speed control handle. Assuming the weight coefficients set are: w1=0.4, w2=0.3, w3=0.3, then the comprehensive fatigue index can be calculated as 61. Among them, the obtained comprehensive fatigue index of 61 can be used to determine whether the driver is in a fatigued state.

[0047] S140. Determine whether the railway driver is driving fatigued according to the comprehensive fatigue index.

[0048] Specifically, the comprehensive fatigue index is a quantitative indicator calculated by integrating the physiological fatigue score, behavioral characteristic fatigue score and driving operation fatigue score. The higher the comprehensive fatigue index, the more serious the driver's fatigue. Judging whether there is fatigue driving means that based on the value of the comprehensive fatigue index, a decision can be made to determine whether the driver is in a fatigue driving state.

[0049] In a possible implementation, judging whether a railway driver is driving in fatigue during driving is performed based on a comprehensive fatigue index, specifically comprising: obtaining historical driving data of the railway driver during historical driving, the historical driving data comprising a marked fatigue driving state and corresponding historical physiological data, historical behavioral characteristic data, and historical driving operation data; performing statistical analysis on the historical driving data using an ROC curve to obtain a fatigue index threshold; comparing the comprehensive fatigue index with the fatigue index threshold; if it is determined that the comprehensive fatigue index is greater than or equal to the fatigue index threshold, determining that the railway driver is driving in fatigue; if it is determined that the comprehensive fatigue index is less than the fatigue index threshold, determining that the railway driver is not driving in fatigue.

[0050] Specifically, historical driving data refers to previously collected data about the driver's driving conditions, which include: annotated fatigue driving status, for example, whether the driver has been driving fatigued for a certain period of time; related historical physiological data, such as heart rate, brain waves, etc.; historical behavioral characteristic data, such as blinking frequency, eye closure time, etc.; historical driving operation data, such as speed control handle rotation, acceleration, etc. The ROC curve is a receiver operating characteristic curve, which is used to evaluate the performance of the classification model and helps determine the optimal classification threshold by depicting the relationship between the true positive rate and the false positive rate. Further, the server collects past driving records about the driver, which include states that have been marked as fatigue or non-fatigue. The fatigue state is analyzed using historical data and the ROC curve is drawn. This process can help researchers identify the optimal dividing point between fatigue and non-fatigue states, that is, the fatigue index threshold. By calculating the true positive rate and false positive rate under various thresholds, it is determined which threshold can best distinguish between fatigue driving and non-fatigue driving. Finally, an optimal fatigue index threshold is obtained from the ROC curve analysis, which will be used for comparison in subsequent judgments. The comprehensive fatigue index calculated in real time is compared with the fatigue index threshold obtained previously. If the comprehensive fatigue index is greater than or equal to the fatigue index threshold, it is judged as fatigue driving. If the comprehensive fatigue index is less than the fatigue index threshold, it is judged as non-fatigue driving.

[0051] In one possible implementation, if it is determined that the comprehensive fatigue index is greater than or equal to the fatigue index threshold, a first-level alarm mechanism is triggered, and the first-level alarm mechanism is used to prompt the railway driver that he is driving fatigued; if it is determined that the comprehensive fatigue index is greater than or equal to the fatigue index threshold, and is greater than or equal to the corresponding preset duration or exceeds a preset number of times, a second-level alarm mechanism is triggered, and the second-level alarm mechanism is used to prompt the railway driver to take a rest and change drivers.

[0052] Specifically, when the comprehensive fatigue index is greater than or equal to the fatigue index threshold, the server will issue a level one alarm to prompt the driver that he has entered a fatigue driving state. If the comprehensive fatigue index not only exceeds the fatigue threshold, but also remains in this state for a certain duration, such as several minutes, or exceeds a preset number of times, such as triggering the level one alarm multiple times, then the level two alarm will be triggered to remind the driver that he must rest or change the driver. Among them, when the level one alarm is triggered, the alarm form may include sound and light alarms, screen prompts, etc. When the level two alarm is triggered, the prompt of the level two alarm will be more urgent, including a strong alarm sound and visual signals, prompting the driver to rest immediately or change the driver.

[0053] For example, assume that the fatigue index threshold is 65, the preset duration is 5 minutes, and the preset number of triggers is 3. The comprehensive fatigue index of a railway driver is calculated to be 68 at a certain moment. Step 1, 68 ≥ 65, so the server triggers a level 1 alarm. Step 2, the server notifies the driver that he is driving fatigued through sound and visual alarms. Assume that in the next monitoring, the comprehensive fatigue index still fluctuates around 68, and the server continues to monitor. After 5 minutes, the comprehensive fatigue index has not dropped below 65 and remains at 68. Because the comprehensive fatigue index has been at 68 for 5 minutes, the server now checks the number of triggers. If the driver has triggered the level 1 alarm 3 times during the past driving process, the server triggers the level 2 alarm. The level 2 alarm will prompt the driver to take a rest immediately or change the driver to ensure driving safety. Through this hierarchical alarm mechanism, the server can respond quickly to the driver's fatigue state to ensure driving safety. The level 2 alarm ensures that the driver takes timely action in a high-risk fatigue state to prevent accidents. By setting a combination of thresholds, duration, and trigger times, the driver's working status can be managed more effectively and the false alarm rate can be reduced.

[0054] In a possible implementation, before inputting the physiological data, the behavioral characteristic data and the driving operation data into the LSTM model to obtain the physiological fatigue score, the behavioral characteristic fatigue score and the driving operation fatigue score, the LSTM model is trained; the LSTM model is trained, referring to Figure 2 , Figure 2 Another flow chart of a method for monitoring fatigue driving of railway drivers provided in an embodiment of the present application specifically includes steps S210 to S250, and the above steps are as follows: S210, obtaining training data; S220, performing missing value processing, data standardization and data slicing on the training data to obtain time series data; S230, performing feature engineering processing on the time series data to obtain an initial model; S240, determining a training set and a validation set based on the training data; S250, optimizing the initial model using the training set and the validation set to obtain an LSTM model.

[0055] Specifically, training data refers to the data set used to train the model, which contains input features and target outputs. There may be missing values ​​in the data, and these missing data are processed by filling, deleting, or interpolating to ensure the quality of model training. Data standardization scales the data to a uniform range, such as 0 to 1 or a mean of 0 and a variance of 1, so that different features have similar influence during training. Data slicing divides time series data into small segments, i.e., time windows, each of which contains a certain number of time steps, to facilitate model learning and prediction. Feature engineering is the process of processing data to extract useful features to help improve model performance, including selecting, modifying, and creating new features. The training set and validation set divide the training data into two parts, one for model training and the other for evaluating model performance.

[0056] Further, the following are the steps for training the LSTM model: By collecting relevant data of railway drivers, including historical physiological data, historical behavioral feature data, and historical driving operation data. Assume that there are several missing records of heart rate in the data set. You can choose to fill these missing values, such as using the average value of the feature, or the average of the previous and next values. Standardize the heart rate data so that all data are between 0 and 1, or adjust its mean to 0 and the variance to 1, so that the influence of different features is more balanced during training. If the data is a continuous time series, such as collecting data once a second, the data can be divided into groups of 10 seconds to form multiple input sequences. For example, obtain the heart rate, blinking frequency, and driving operation data within 10 seconds as an input. In the above time window, statistical features such as average heart rate and maximum blinking frequency can be extracted. These features will be used as inputs to the LSTM model. The processed data set is divided into a training set, such as 80% of the data, and a validation set, such as 20% of the data, for training and evaluating the performance of the model. The LSTM model is trained using the training set, and its performance is evaluated on the validation set. Model parameters such as learning rate and number of layers are adjusted to improve monitoring accuracy. By acquiring and preprocessing training data and converting it into a format suitable for the LSTM model, the model's predictive ability can be improved, thereby effectively monitoring the fatigue status of railway drivers and providing support for safe driving.

[0057] The present application also provides a monitoring device for railway driver fatigue driving, referring to Figure 3 , Figure 3A module schematic diagram of a monitoring device for fatigue driving of a railway driver provided in an embodiment of the present application. The monitoring device is a server, and the server includes an acquisition module 31 and a processing module 32, wherein the acquisition module 31 acquires the physiological data, behavioral characteristic data and driving operation data of the railway driver during driving, the physiological data includes heart rate variability, brain wave characteristics and skin electrical response, the behavioral characteristic data includes blinking frequency, eye closing time, gaze point change and yawning number, and the driving operation data includes the rotation angle of the speed control handle, acceleration change and braking frequency; the processing module 32 inputs the physiological data, behavioral characteristic data and driving operation data into the LSTM model to obtain the physiological fatigue score, behavioral characteristic fatigue score and driving operation fatigue score; the processing module 32 determines the comprehensive fatigue index based on the physiological fatigue score, behavioral characteristic fatigue score and driving operation fatigue score; the processing module 32 determines whether the railway driver is driving fatigued during driving according to the comprehensive fatigue index.

[0058] In a possible implementation, the acquisition module 31 acquires the physiological data, behavioral characteristic data and driving operation data of the railway driver during the driving process, specifically including: the acquisition module 31 receives the original physiological data sent by the smart wearable device, the railway driver wears the smart wearable device, and the smart wearable device includes a smart watch and an EEG headband; the acquisition module 31 receives the cab image data sent by the cab monitoring device; the processing module 32 uses an image processing algorithm to perform facial image processing on the cab image data to obtain the original behavioral characteristic data; the acquisition module 31 receives the CAN bus data sent by the on-board sensor device, and the on-board sensor device includes a speed control handle sensor device, an acceleration sensor device and a brake pressure sensor device; the processing module 32 classifies and processes the CAN bus data to obtain the original driving operation data; the processing module 32 obtains the physiological data, the behavioral characteristic data and the driving operation data by pre-processing the original physiological data, the original behavioral characteristic data and the original driving operation data.

[0059] In a possible implementation, the processing module 32 inputs the physiological data, the behavioral characteristic data, and the driving operation data into the LSTM model to obtain the physiological fatigue score, the behavioral characteristic fatigue score, and the driving operation fatigue score, specifically using the following calculation formula: ; in, is the physiological fatigue score, is the individual weight of the physiological data, is a nonlinear activation function, is the value of the ith physiological characteristic, is the time attenuation factor of the physiological characteristic, t is the time, For physiological characteristics and physiological characteristics The interaction weight between is the value of the jth physiological characteristic; ; in, is the behavioral characteristic fatigue score, is the individual weight of the behavioral feature data, is a nonlinear activation function, is the value of the kth behavior feature, is the time decay factor of the behavior characteristic, t is the time, Behavioral characteristics and behavioral characteristics The interaction weight between is the value of the lth behavior feature; ; in, is the driving operation fatigue score, is the individual weight of the driving operation data, is a nonlinear activation function, is the value of the pth driving operation feature, is the time attenuation factor of driving operation characteristics, t is time, Driving operation characteristics and driving operation characteristics The interaction weight between is the value of the qth driving operation feature.

[0060] In a possible implementation, the processing module 32 determines the comprehensive fatigue index based on the physiological fatigue score, the behavioral characteristic fatigue score, and the driving operation fatigue score, specifically using the following calculation formula: ; in, is the comprehensive fatigue index, is the overall weight of physiological data, is the overall weight of the behavioral feature data, is the overall weight of the driving operation data.

[0061] In a possible implementation, the processing module 32 determines whether the railway driver is driving in fatigue according to the comprehensive fatigue index, specifically including: the acquisition module 31 acquires the historical driving data of the railway driver during the historical driving process, the historical driving data including the marked fatigue driving state and the corresponding historical physiological data, historical behavioral characteristic data and historical driving operation data; the processing module 32 uses the ROC curve to perform statistical analysis on the historical driving data to obtain the fatigue index threshold; the processing module 32 compares the comprehensive fatigue index with the fatigue index threshold; if the processing module 32 determines that the comprehensive fatigue index is greater than or equal to the fatigue index threshold, it determines that the railway driver is driving in fatigue; if the processing module 32 determines that the comprehensive fatigue index is less than the fatigue index threshold, it determines that the railway driver is not driving in fatigue.

[0062] In one possible implementation, if the processing module 32 determines that the comprehensive fatigue index is greater than or equal to the fatigue index threshold, a first-level alarm mechanism is triggered, and the first-level alarm mechanism is used to prompt the railway driver that he is driving fatigued; if the processing module 32 determines that the comprehensive fatigue index is greater than or equal to the fatigue index threshold, and is greater than or equal to the corresponding continuous preset time or exceeds the preset number of times, a second-level alarm mechanism is triggered, and the second-level alarm mechanism is used to prompt the railway driver to take a rest and change drivers.

[0063] In a possible implementation, the processing module 32 trains the LSTM model before inputting the physiological data, behavioral characteristic data and driving operation data into the LSTM model to obtain the physiological fatigue score, the behavioral characteristic fatigue score and the driving operation fatigue score; the training of the LSTM model specifically includes: the acquisition module 31 acquires the training data; the processing module 32 performs missing value processing, data standardization and data slicing on the training data to obtain time series data; the processing module 32 performs feature engineering on the time series data to obtain an initial model; the processing module 32 determines the training set and the validation set based on the training data; the processing module 32 uses the training set and the validation set to optimize the initial model to obtain the LSTM model.

[0064] It should be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0065] The present application also provides an electronic device, referring to Figure 4 , Figure 4The electronic device may include: at least one processor 41 , at least one network interface 44 , a user interface 43 , a memory 45 , and at least one communication bus 42 .

[0066] The communication bus 42 is used to realize the connection and communication between these components.

[0067] The user interface 43 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 43 may also include a standard wired interface and a wireless interface.

[0068] The network interface 44 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0069] Among them, the processor 41 may include one or more processing cores. The processor 41 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 45, and calling data stored in the memory 45. Optionally, the processor 41 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 41 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 41, and it can be implemented separately through a chip.

[0070] Among them, the memory 45 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory 45 includes a non-transitory computer-readable storage medium. The memory 45 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 45 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store data involved in the above-mentioned method embodiments, etc. The memory 45 may also be optionally at least one storage device located away from the aforementioned processor 41. As Figure 4 As shown, the memory 45 as a computer storage medium may include an operating system, a network communication module, a user interface module and an application program of a method for monitoring fatigue driving of railway drivers.

[0071] exist Figure 4 In the electronic device shown, the user interface 43 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 41 can be used to call an application program stored in the memory 45 for a method for monitoring fatigue driving of a railway driver. When executed by one or more processors, the electronic device executes one or more methods in the above-mentioned embodiments.

[0072] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0073] The present application also provides a computer-readable storage medium, which stores instructions. When executed by one or more processors, the electronic device executes one or more of the methods described in the above embodiments.

[0074] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0075] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0076] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0077] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0078] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0079] The above is only an exemplary embodiment of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, it will be easy for those skilled in the art to think of other embodiments of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field that are not recorded in the present disclosure. The description and examples are regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for monitoring railway driver fatigue driving, characterized in that: The method comprises: Acquiring physiological data, behavioral characteristic data and driving operation data of the railway driver during driving, wherein the physiological data includes heart rate variability, brain wave characteristics and skin electrical response, the behavioral characteristic data includes blinking frequency, eye closing time, gaze point change and yawning frequency, and the driving operation data includes speed control handle rotation angle, acceleration change and braking frequency; Inputting the physiological data, the behavioral characteristic data and the driving operation data into the LSTM model to obtain a physiological fatigue score, a behavioral characteristic fatigue score and a driving operation fatigue score; determining a comprehensive fatigue index based on the physiological fatigue score, the behavioral characteristic fatigue score, and the driving operation fatigue score; According to the comprehensive fatigue index, it is determined whether the railway driver is driving fatigued during the driving process.

2. The method for monitoring railway driver fatigue driving according to claim 1, characterized in that: The acquisition of physiological data, behavioral characteristic data and driving operation data of the railway driver during driving specifically includes: Receiving raw physiological data sent by a smart wearable device, the railway driver is wearing the smart wearable device, the smart wearable device includes a smart watch and an EEG headband; Receiving cab image data sent by cab monitoring equipment; Using an image processing algorithm to perform facial image processing on the cab image data to obtain original behavior feature data; Receiving CAN bus data sent by an on-board sensor device, wherein the on-board sensor device includes a speed control handle sensor device, an acceleration sensor device, and a brake pressure sensor device; Classifying and processing the CAN bus data to obtain original driving operation data; The physiological data, the behavioral characteristic data and the driving operation data are obtained by preprocessing the original physiological data, the original behavioral characteristic data and the original driving operation data.

3. The method for monitoring railway driver fatigue driving according to claim 1, characterized in that: The physiological data, the behavioral characteristic data and the driving operation data are all input into the LSTM model to obtain the physiological fatigue score, the behavioral characteristic fatigue score and the driving operation fatigue score, specifically using the following calculation formula: ； in, is the physiological fatigue score, is the individual weight of the physiological data, is a nonlinear activation function, is the value of the ith physiological characteristic, is the time attenuation factor of the physiological characteristic, t is the time, For physiological characteristics and physiological characteristics The interaction weight between is the value of the jth physiological characteristic; ； in, is the behavioral characteristic fatigue score, is the individual weight of the behavioral feature data, is a nonlinear activation function, is the value of the kth behavior feature, is the time decay factor of the behavior characteristic, t is the time, Behavioral characteristics and behavioral characteristics The interaction weight between is the value of the lth behavior feature; ； in, is the driving operation fatigue score, is the individual weight of the driving operation data, is a nonlinear activation function, is the value of the pth driving operation feature, is the time attenuation factor of driving operation characteristics, t is time, Driving operation characteristics and driving operation characteristics The interaction weight between is the value of the qth driving operation feature.

4. The method for monitoring railway driver fatigue driving according to claim 3 is characterized in that: The comprehensive fatigue index is determined based on the physiological fatigue score, the behavioral characteristic fatigue score and the driving operation fatigue score, and specifically adopts the following calculation formula: ； in, is the comprehensive fatigue index, is the overall weight of physiological data, is the overall weight of the behavioral feature data, is the overall weight of the driving operation data.

5. The method for monitoring railway driver fatigue driving according to claim 1, characterized in that: The determining, based on the comprehensive fatigue index, whether the railway driver is driving fatigued during driving specifically includes: Acquiring historical driving data of the railway driver during historical driving, wherein the historical driving data includes a marked fatigue driving state and corresponding historical physiological data, historical behavioral characteristic data, and historical driving operation data; Performing statistical analysis on the historical driving data using a ROC curve to obtain a fatigue index threshold; comparing the comprehensive fatigue index with the fatigue index threshold; If it is determined that the comprehensive fatigue index is greater than or equal to the fatigue index threshold, then the railway driver is determined to be driving fatigued; If it is determined that the comprehensive fatigue index is less than the fatigue index threshold, it is determined that the railway driver is not driving fatigued.

6. The method for monitoring railway driver fatigue driving according to claim 5, characterized in that: The method further comprises: If it is determined that the comprehensive fatigue index is greater than or equal to the fatigue index threshold, a first-level alarm mechanism is triggered, and the first-level alarm mechanism is used to prompt the railway driver that he is driving fatigued; If it is determined that the comprehensive fatigue index is greater than or equal to the fatigue index threshold, and is greater than or equal to the corresponding preset duration or exceeds the preset number of times, a secondary alarm mechanism is triggered, and the secondary alarm mechanism is used to prompt the railway driver to take a rest and change drivers.

7. The method for monitoring railway driver fatigue driving according to claim 1, characterized in that: Before inputting the physiological data, the behavioral characteristic data and the driving operation data into the LSTM model to obtain the physiological fatigue score, the behavioral characteristic fatigue score and the driving operation fatigue score, the LSTM model is trained; the training of the LSTM model specifically includes: Get training data; Performing missing value processing, data standardization, and data slicing on the training data to obtain time series data; Performing feature engineering processing on the time series data to obtain an initial model; Determine a training set and a validation set based on the training data; The initial model is optimized using the training set and the validation set to obtain the LSTM model.

8. A monitoring device for railway driver fatigue driving, characterized in that: The monitoring device comprises an acquisition module (31) and a processing module (32), wherein: The acquisition module (31) is used to acquire physiological data, behavioral characteristic data and driving operation data of the railway driver during driving, wherein the physiological data includes heart rate variability, brain wave characteristics and skin electrical response, the behavioral characteristic data includes blinking frequency, eye closing time, gaze point change and yawning frequency, and the driving operation data includes speed control handle rotation angle, acceleration change and braking frequency; The processing module (32) is used to input the physiological data, the behavioral characteristic data and the driving operation data into the LSTM model to obtain a physiological fatigue score, a behavioral characteristic fatigue score and a driving operation fatigue score; The processing module (32) is further used to determine a comprehensive fatigue index based on the physiological fatigue score, the behavioral characteristic fatigue score and the driving operation fatigue score; The processing module (32) is also used to determine whether the railway driver is driving fatigued during the driving process based on the comprehensive fatigue index.

9. An electronic device, characterized in that: The electronic device comprises a processor (41), a memory (45), a user interface (43) and a network interface (44), wherein the memory (45) is used to store instructions, the user interface (43) and the network interface (44) are both used to communicate with other devices, and the processor (41) is used to execute the instructions stored in the memory (45) so that the electronic device executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is performed.

Images