Driver fatigue monitoring system based on reinforcement learning

Abstract

The invention relates to the technical field of mine card driving safety monitoring, and discloses a driver fatigue monitoring system based on reinforcement learning. A multi-source data acquisition module of the system acquires physiological signal data and video image data of a driver in real time; the layered feature extraction module analyzes a plurality of different levels of fatigue state features such as eye movement, head posture and body behavior from the data; the dynamic feature aggregation module performs progressive fusion processing on the features to generate comprehensive fatigue feature representation; the historical mode retrieval module obtains a similar historical fatigue mode by calculating the similarity between the comprehensive fatigue feature representation and the feature representation in the historical fatigue feature library; the reinforcement learning reasoning module inputs the comprehensive fatigue feature representation and the similar historical fatigue mode into a reinforcement learning model, and outputs a driver real-time fatigue prediction value; the fatigue state judgment module judges whether the driver is in a fatigue state or not according to a comparison result of the predicted value and a preset threshold value.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mine truck driving safety monitoring, in particular to a driver fatigue monitoring system based on reinforcement learning. BACKGROUND

[0002] In the field of mine transportation, mine trucks serve as the main transportation tool and bear heavy material transportation tasks. However, mine truck drivers are in a high-intensity and high-pressure working environment for a long time, and fatigue driving phenomena occur frequently, which poses a great threat to the safety of personnel life and property. The technologies on the market for detecting fatigue of mine truck drivers mainly include physiological signal detection and facial recognition. The technology based on physiological signal detection usually collects physiological signals such as heart rate and brain waves of the driver through sensors worn on the driver's body, such as electrocardiogram sensors and electroencephalogram sensors. The principle is that when the driver is in a fatigue state, the physiological signals will change accordingly, for example, the heart rate variability increases, the frequency and amplitude of brain waves change, etc., and the fatigue of the driver is judged by analyzing the changes of these physiological signals. This technology is generally applied to scenes requiring high fatigue state of the driver and requiring accurate monitoring, such as transportation operations in some large open-pit mines. The fatigue detection technology based on facial recognition uses a camera to collect the facial image of the driver, analyzes facial features such as the degree of eye closure, blink frequency, yawning frequency, and head posture to determine the fatigue state of the driver. For example, the degree of eye closure is determined by calculating the eye aspect ratio (EAR), and when the EAR value is below a certain threshold and lasts for a period of time, it is determined that the driver may be in a fatigue state; abnormal postures such as nodding and shaking of the head are detected to assist in determining fatigue. This technology is widely used in various mine trucks due to its non-contact, easy installation and use, etc.

[0003] Although the existing fatigue detection technology can monitor the fatigue state of the mine truck driver to some extent, there are still many limitations in practical application. In terms of data collection, the existing technology can only collect a single type of data, such as only collecting physiological signal data or only collecting facial image data, and cannot comprehensively obtain the state information of the driver. Taking the physiological signal detection technology as an example, although it can accurately reflect the physiological changes of the driver, the sensor is inconvenient to wear, which may affect the normal operation of the driver, and the data collection is easily disturbed by motion, resulting in inaccurate data. When the light changes, the driver's face is blocked (such as wearing a mask, a safety helmet blocks part of the face), the quality of the collected image will decrease, and even effective images cannot be collected, which seriously affects the detection result. In terms of the accuracy of feature extraction, the existing technology has difficulty in accurately extracting multiple different levels of features when analyzing the fatigue state features, and the correlation analysis between the features is not deep enough. For example, based on facial recognition technology, in complex environments, the extraction of eye movement features, head posture features and body behavior features is prone to deviation, resulting in inaccurate judgment of the fatigue state. In terms of the reliability of fatigue state judgment, the current technology is mostly based on fixed models and thresholds for judgment, and lacks adaptability to individual differences of different drivers and different working scenarios. Each driver has different fatigue performance and physiological characteristics, and the fixed judgment standard cannot accurately identify the fatigue state of all drivers. Moreover, the working scenarios of the mine are complex and changeable, such as different working time, road conditions, transportation tasks, etc., and the existing detection technology is difficult to dynamically adjust according to the actual situation, resulting in low reliability of fatigue state judgment. Therefore, it is of great practical significance to develop a more comprehensive, accurate and reliable mine truck driver fatigue warning system. SUMMARY

[0004] The present application aims to provide a driver fatigue monitoring system based on reinforcement learning to solve the problems raised in the background art.

[0005] To achieve the above-mentioned purpose, the present application provides a driver fatigue monitoring system based on reinforcement learning, which comprises: a multi-source data collection module for collecting physiological signal data and video image data of the mine truck driver in real time; a hierarchical feature extraction module for analyzing a plurality of different levels of fatigue state features from the physiological signal data and video image data, the fatigue state features including eye movement features, head posture features and body behavior features; a dynamic feature aggregation module for performing progressive fusion processing on the plurality of different levels of fatigue state features to generate a comprehensive fatigue feature representation; a historical pattern retrieval module, configured to calculate a similarity between the comprehensive fatigue feature representation and a feature representation stored in a historical fatigue feature library, and obtain a similar historical fatigue pattern according to a similarity result; a reinforcement learning inference module, configured to input the comprehensive fatigue feature representation and the similar historical fatigue pattern into a reinforcement learning model, and output a real-time fatigue degree prediction value of the driver; a fatigue state determination module, configured to determine whether the driver is in a fatigue state according to a comparison result of the real-time fatigue degree prediction value of the driver and a preset threshold.

[0006] Preferably, the dynamic feature aggregation module is specifically configured to: identify an initial feature set of a basic level from the fatigue state features of the multiple different levels; perform normalization processing on the initial feature set to generate a standardized feature set; perform weighted combination on the features in the standardized feature set to generate a primary fusion feature; identify an advanced feature set of an intermediate level from the fatigue state features of the multiple different levels; perform splicing operation on the advanced feature set and the primary fusion feature to generate an intermediate fusion feature; identify an abstract feature set of a high level from the fatigue state features of the multiple different levels; perform attention weighted fusion on the abstract feature set and the intermediate fusion feature to generate the comprehensive fatigue feature representation.

[0007] Preferably, the normalization processing performed by the dynamic feature aggregation module on the initial feature set includes: calculating a mean value and a variance of each feature in the initial feature set; performing standardization transformation on each feature based on the mean value and the variance, so that the feature value distribution is within a preset range; performing linear scaling on the standardized features to generate the standardized feature set.

[0008] Preferably, the weighted combination performed by the dynamic feature aggregation module on the standardized feature set includes: assigning a trainable weight parameter to each feature in the standardized feature set; multiplying each feature by the corresponding weight parameter to obtain a weighted feature; performing summation operation on all weighted features to generate the primary fusion feature.

[0009] Preferably, the splicing operation performed by the dynamic feature aggregation module on the advanced feature set and the primary fusion feature includes: concatenate the feature vectors in the advanced feature set with the primary fusion feature vector in feature dimensions; perform dimension reduction processing on the concatenated feature vectors to reduce feature dimensions to a preset size; apply a nonlinear activation function to transform the dimension-reduced feature vectors to generate the intermediate fusion features.

[0010] Preferably, the hierarchical feature extraction module is specifically configured to: perform filtering and denoising processing on the physiological signal data to extract time-domain and frequency-domain features; perform frame sequence analysis on the video image data to detect facial key points and body joint points of the driver; calculate eyelid closure frequency and pupil change rate based on the facial key points to generate the eye movement features; calculate head rotation angle and nodding frequency based on the facial key points to generate the head posture features; calculate limb movement amplitude and posture change period based on the body joint points to generate the body behavior features.

[0011] Preferably, the historical pattern retrieval module is specifically configured to: load all historical fatigue feature representations from the historical fatigue feature library; calculate cosine similarity values between the comprehensive fatigue feature representation and each historical fatigue feature representation; sort the historical fatigue feature representations according to the cosine similarity values and select top-k most similar feature representations; perform weighted averaging on the top-k most similar feature representations to generate the similar historical fatigue pattern.

[0012] Preferably, the system further comprises a historical fatigue feature library construction module configured to: collect multi-source sensor data and corresponding fatigue event labels in historical driving sessions; perform feature extraction on the data of each historical driving session to generate historical fatigue state features; store the historical fatigue state features and fatigue event labels in association in a database; periodically update the database to add newly collected data and labels.

[0013] Preferably, the reinforcement learning inference module is specifically configured to: combine the comprehensive fatigue feature representation and the similar historical fatigue pattern into a state vector; input the state vector into a pre-trained deep Q network model; The Q value of each possible action is calculated through forward propagation of the deep Q network model; The action with the maximum Q value is selected as the fatigue degree prediction strategy, and the real-time fatigue degree prediction value of the driver is output.

[0014] Preferably, the fatigue state determination module is specifically used for: A plurality of fatigue degree thresholds are set, including an attention threshold and a danger threshold; When the real-time fatigue degree prediction value of the driver exceeds the attention threshold but is lower than the danger threshold, it is determined that the driver is in a low fatigue state, and a low-level warning signal is generated to trigger a visual reminder; When the real-time fatigue degree prediction value of the driver exceeds the danger threshold, it is determined that the driver is in a high fatigue state, and a high-level warning signal is generated, and visual and auditory reminders are triggered, and the reminder intensity and duration are adjusted according to the warning type.

[0015] Compared with the prior art, the present application has the following advantages: The driver fatigue monitoring system based on reinforcement learning has significant advantages in ensuring the safety of mine truck driving. The multi-source data acquisition module of the system can acquire physiological signal data and video image data of the mine truck driver in real time. Compared with the traditional single data acquisition technology, this comprehensive data acquisition method can obtain the state information of the driver from multiple dimensions, avoiding the information loss caused by single data. For example, through the physiological signal data, the changes of the driver's heart rate, brain wave and other physiological indicators can be understood, and the video image data can intuitively reflect the driver's facial expressions, body posture and other conditions, providing a rich and comprehensive data basis for subsequent fatigue detection.

[0016] The hierarchical feature extraction module analyzes a plurality of fatigue state features of different levels from the collected physiological signal data and video image data, covering eye movement features, head posture features and body behavior features and other multiple dimensions. Unlike the simple extraction of single features in the prior art, this multi-dimensional feature extraction can more comprehensively and deeply reflect the fatigue state of the driver. Taking the eye movement feature as an example, not only the blinking frequency can be detected, but also the amplitude and speed of the eyeball rotation can be analyzed; the head posture feature extraction can accurately identify nodding, shaking, lowering the head and other abnormal postures; the body behavior feature focuses on the changes of the driver's sitting posture, body shaking and other conditions. These multi-dimensional features complement each other, greatly improving the accuracy of fatigue state judgment.

[0017] The dynamic feature aggregation module gradually fuses multiple fatigue state features of different levels to generate a comprehensive fatigue feature representation. This fusion method can fully exploit the internal relationship between different features, making the comprehensive fatigue feature more accurately reflect the true fatigue state of the driver. Compared with the traditional simple splicing or weighted sum method, the gradual fusion process considers the importance and interaction of features, which can effectively avoid judgment errors caused by feature conflicts or redundancy, thereby improving the reliability of fatigue state judgment.

[0018] The historical pattern retrieval module calculates the similarity between the comprehensive fatigue feature representation and the feature representations stored in the historical fatigue feature library, and obtains similar historical fatigue patterns according to the similarity results. This process provides a historical data reference for fatigue state judgment, which can learn from past experience to better identify the current driver's fatigue state. In the face of complex and variable driving scenarios and individual differences of different drivers, the historical pattern retrieval module can quickly match similar situations to provide strong support for accurate fatigue state judgment, making up for the lack of historical data reference and single judgment standard in the prior art.

[0019] The reinforcement learning inference module inputs the comprehensive fatigue feature representation and the similar historical fatigue pattern into the reinforcement learning model, and outputs the real-time fatigue degree prediction value of the driver. The reinforcement learning model has strong learning and reasoning ability, which can be self-optimized and adjusted according to the continuously updated data, thereby more accurately predicting the real-time fatigue degree of the driver. Compared with the traditional fixed model prediction method, the reinforcement learning inference module can adapt to different driving environments and changes of drivers, providing more accurate and timely fatigue degree prediction to gain valuable time for early warning measures.

[0020] The fatigue state determination module generates and triggers a warning signal according to the comparison result of the real-time fatigue degree prediction value of the driver and the preset threshold. This dynamic warning method according to the actual fatigue degree can timely remind the driver to pay attention to the fatigue state and avoid accidents caused by fatigue driving. When the driver's fatigue degree approaches or exceeds the preset threshold, the system will immediately issue a warning signal such as a sound prompt, a vibration reminder, etc., effectively attracting the driver's attention and prompting him to take measures such as rest, thereby reducing the risk of accidents. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 The working principle diagram of the driver fatigue monitoring system based on reinforcement learning described in the present application; Figure 2 The working flowchart of the dynamic feature aggregation module; Figure 3 The flowchart of the initial feature set normalization processing of the dynamic feature aggregation module; Figure 4The trend chart of the multi-dimensional features of the fatigue state of the mine truck driver changing with the working time; Figure 5 The comparative chart of the response of the different warning levels of the mine truck driver fatigue warning system. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0023] Please refer to Figure 1 The present application provides a driver fatigue monitoring system based on reinforcement learning, which comprises: a multi-source data acquisition module continuously acquires physiological signal data and video image data of a mine truck driver, the physiological signal data including electrocardiogram, electroencephalogram and other bioelectric signals, and the video image data including video streams of the face and body of the driver; a hierarchical feature extraction module analyzes eye movement features, head posture features and body behavior features and other multi-layer fatigue state features from the original data, which reflect different fatigue dimensions of the driver from micro physiological changes to macro behavior performance; a dynamic feature aggregation module adopts a progressive fusion strategy to integrate different hierarchical features into comprehensive fatigue feature representation, ensuring the integrity and representativeness of the feature information; a historical pattern retrieval module calculates the similarity between the comprehensive fatigue feature representation and the stored patterns in the historical fatigue feature library, and retrieves similar historical fatigue patterns as references; a reinforcement learning inference module inputs real-time features and historical patterns into a deep Q network model, and outputs real-time fatigue degree prediction values of the driver; and a fatigue state determination module determines whether the driver is in a fatigue state according to the comparison result of the prediction values and preset threshold values, and triggers corresponding level warning signals to realize graded warning response.

[0024] Embodiment 1: refer to Figure 2, the dynamic feature aggregation module of the driver fatigue monitoring system based on reinforcement learning performs a progressive fusion operation when processing multiple different levels of fatigue state features. The module identifies an initial feature set of a basic level from multiple different levels of fatigue state features, and the basic level features include eyelid closure frequency in eye movement features and head rotation angle in head posture features, which are low-level physical indicators. The initial feature set is derived from the output of the hierarchical feature extraction module and contains basic signal attributes directly parsed from raw sensor data. These attributes have clear physical meaning and lower abstraction levels. Eyelid closure frequency represents the number of times the driver's eyes close per unit time, and head rotation angle reflects the degree of head deflection relative to the forward direction. These feature values are calculated from video stream data through algorithms, and are usually continuous scalar or short time series in numerical form. The identification process of the initial feature set relies on pre-set feature classification rules, which define which features belong to the basic level. The classification criteria are based on the complexity and calculation source of the features. The basic level features are usually primary indicators calculated directly from the original data in one step. The initial feature set is normalized to generate a standardized feature set. The normalization process calculates the mean and variance of each feature in the initial feature set. The mean represents the center position of the feature in the historical data distribution, and the variance describes the range of discrete fluctuations of the feature value. The calculation of mean and variance is based on the feature samples within a sliding time window. The window size is set according to the real-time requirements of the system, usually covering several minutes of feature data. Each feature is standardized based on the mean and variance. The transformation formula subtracts the mean from each feature value and divides it by the variance, so that the transformed feature value distribution is within the range of zero mean and unit variance. This standardization eliminates the dimensional differences and numerical scale inconsistencies between different features. The standardized features are linearly scaled to generate a standardized feature set. Linear scaling maps the standardized values to a fixed interval, such as zero to one, further constraining the range of feature values and avoiding interference from extreme outliers in subsequent fusion steps.

[0025] The features in the standardized feature set are combined by weighting to generate the primary fusion feature. Each feature in the standardized feature set is assigned a trainable weight parameter, which is a floating-point number. The initial value of the weight parameter is randomly initialized, and is updated by the back propagation algorithm and gradient descent method in the system training stage. Each feature is multiplied by the corresponding weight parameter to obtain a weighted feature. The multiplication operation amplifies the contribution of the feature with high importance and suppresses the influence of the feature with low importance. The size of the weight implicitly represents the importance level of different basic features for fatigue state discrimination. The weighted features are summed to generate the primary fusion feature. The summation operation combines the weighted features into a single scalar value, which represents the fusion result of the basic level features and is a compact numerical representation of the low-level feature information. The weighting combination process is essentially a linear transformation that projects the multi-dimensional basic feature vector onto a one-dimensional fusion feature. The primary fusion feature, as a summary representation of the basic level features, is input to the subsequent fusion stage. The dynamic feature aggregation module identifies an advanced feature set from multiple fatigue state features at different levels. The advanced feature set includes intermediate abstract indicators such as limb movement amplitude in the body behavior feature. The advanced features have a higher degree of abstraction than the basic features. They are usually not directly calculated from the original data, but are derived by further statistical, transformation or combination of the basic features. The limb movement amplitude feature is calculated by analyzing the change trajectory of the driver's joint coordinates in consecutive video frames. It reflects the overall activity level of the driver's upper body and arms, and is a comprehensive behavior indicator. The identification of the advanced feature set is also based on the pre-set classification rules. The rules classify features that require multi-step calculation or contain certain temporal context information as intermediate level features.

[0026] The advanced feature set is spliced with the primary fusion feature to generate a middle-level fusion feature. The splicing operation connects the feature vectors in the advanced feature set and the primary fusion feature vector in the feature dimension. For example, if the advanced feature is a five-dimensional vector and the primary fusion feature is a scalar, a six-dimensional composite feature vector is generated after splicing. The connected feature vector is processed by dimension reduction to reduce the feature dimension to a preset size. The dimension reduction operation is implemented through a fully connected neural network layer. The network layer has a fixed input dimension and a small output dimension. Its function is to compress the high-dimensional spliced vector into a low-dimensional representation space. The dimension reduction processing eliminates the redundant information in the feature vector and retains the most important feature components, which helps to improve the generalization ability and computational efficiency of the model. A nonlinear activation function is applied to transform the dimension-reduced feature vector to generate a middle-level fusion feature. The nonlinear activation function, such as the ReLU function, is applied to the output of the fully connected layer. The nonlinear transformation capability is introduced to enable the model to learn the complex interaction relationship between features. The splicing and dimension reduction operations effectively integrate the feature information of the base layer and the intermediate layer. The middle-level fusion feature contains both the low-level detail information and the middle-level context information.

[0027] The dynamic feature aggregation module identifies a high-level abstract feature set from a plurality of fatigue state features at different levels, the abstract feature set including high-level semantic features such as posture change period extracted from video image data. The abstract feature set is at the highest level of the feature hierarchy, and has the strongest semantic information, which usually requires complex models such as deep learning networks to extract from raw data. The posture change period feature analyzes the cyclic change pattern of the driver's body posture in a long time window, which requires the use of a time series model such as LSTM to model the sequence data of the joint nodes. The identification of the abstract feature set relies on a pre-defined feature level mapping table to ensure that only those features with global semantics extracted by deep networks are included in this category. The abstract feature set and the intermediate fusion features are attention-weighted fused to generate a comprehensive fatigue feature representation. The attention-weighted fusion mechanism first calculates the correlation weight between each feature in the abstract feature set and the intermediate fusion features. The correlation weight is calculated by a small neural network attention module, which takes the abstract feature and the intermediate fusion feature as input and outputs a weight value between zero and one. The size of the weight value represents the degree of association between the abstract feature and the current intermediate fusion feature, and the larger the weight, the more important the abstract feature contributes to the final comprehensive representation. Each feature in the abstract feature set is multiplied by its corresponding attention weight and then summed to obtain the weighted abstract feature representation. Finally, this weighted abstract feature representation is element-wise added to the intermediate fusion feature to generate the final comprehensive fatigue feature representation. The attention mechanism allows the model to dynamically adjust the importance of different high-level features, enabling the fusion process to focus on the high-level semantic information most relevant to the current fatigue state. The comprehensive fatigue feature representation, as a fixed-dimensional vector, encapsulates fatigue state information from low to high levels, providing a comprehensive and discriminative state input for subsequent historical pattern retrieval and reinforcement learning reasoning. The calculation process of the entire dynamic feature aggregation module is designed to be differentiable, facilitating end-to-end training and optimization with other modules in the system. The implementation of the module on embedded hardware takes into account the constraints of computing resources, using an optimized matrix operation library to ensure the real-time performance of the feature aggregation process, meeting the low-latency requirements of fatigue warning in long-term continuous operation of mining cards. The intermediate results in the feature aggregation process are cached in memory for error backpropagation and possible diagnostic analysis within the module. The module provides a standardized data interface to receive feature inputs from the hierarchical feature extraction module and outputs comprehensive fatigue feature representations to the historical pattern retrieval module.

[0028] Embodiment 2: see Figure 3The dynamic feature aggregation module adopts a standardization transformation method for normalizing the initial feature set, calculates the mean and variance of each feature in the initial feature set, the mean represents the average level of the feature value in the time series, and the variance reflects the fluctuation degree of the feature value around the mean. The mean is calculated by the arithmetic mean method, and the sum of all feature sample values in the sliding time window is divided by the number of samples. The variance calculation is based on the average of the square difference between the sample value and the mean. The length of the sliding time window is set according to the real-time requirement of the mine truck driving, and the window length needs to balance the calculation timeliness and statistical reliability. A longer window can provide more stable statistics but will increase the calculation delay. Based on the mean and variance, each feature is standardized and transformed, and the transformation process subtracts the mean of each original feature value from the variance. This linear transformation makes the transformed feature value distribution have the characteristics of zero mean and unit variance. The standardization transformation eliminates the scale difference caused by the different dimensions and original value ranges of different features, so that all features are in a comparable numerical range, laying a foundation for subsequent feature weighted combination. The standardized features are linearly scaled to generate a standardized feature set. Linear scaling maps the standardized feature value to a closed interval between zero and one, and the scaling formula uses the minimum-maximum scaling method, using the preset global minimum and maximum as the scaling boundary. Linear scaling further unifies the distribution range of the feature value, avoiding the negative impact of extreme values on model training. The standardized feature set is the final output of the normalization process and enters the weighted combination stage.

[0029] The dynamic feature aggregation module assigns trainable weight parameters when combining the standardized feature set, the weight parameters are model parameters in the form of floating-point numbers, each feature corresponds to an independent weight parameter, and all weight parameters form a weight vector. The initial value of the weight parameter adopts a random initialization strategy, which is randomly sampled from a normal distribution with a mean of zero and a small variance to break the initial symmetry. The trainable weight parameters are updated during the model training process through the back propagation algorithm and the gradient descent optimization method. The gradient descent algorithm adjusts the weight value according to the gradient direction of the loss function with respect to the weight parameter, and gradually optimizes the weight configuration. Each feature is multiplied by the corresponding weight parameter to obtain a weighted feature, the multiplication operation is element-by-element multiplication, and each numerical component in the feature vector is multiplied by the weight value at the corresponding position in the weight vector to produce a weighted feature vector of the same dimension. The weighting operation realizes the importance adjustment of the feature, the feature with a larger absolute weight value occupies a larger proportion in the subsequent fusion, the feature with a smaller absolute weight value has a weakened influence, and the weight sign determines the promotion or inhibition direction of the feature to the fusion result. The sum operation is performed on all weighted features to generate a primary fusion feature, and the sum operation adds all components in the weighted feature vector to produce a scalar value as the primary fusion feature. The sum operation realizes the aggregation of feature information, compresses the multi-dimensional feature into a single-dimensional numerical representation, and the primary fusion feature is a compact representation of the base-level feature, which retains the main information of the original feature. The weighted combination process is essentially a linear transformation, which learns the relative importance of different features to the fatigue state discrimination through the weight parameter, and the primary fusion feature is an intermediate result of the feature aggregation process, which is input to the next stage of feature fusion operation.

[0030] The normalization processing module is implemented in a batch processing manner in the embedded system, and the system maintains a ring buffer for each feature to store the feature values in the recent period of time, and the buffer length is consistent with the size of the sliding time window. When a new feature value arrives, the system updates the buffer content and recalculates the mean and variance, and uses a recursive calculation method to reduce the calculation overhead and avoid recalculating all historical data every time. The mean recursive calculation updates the new mean value using the old mean value and the new sample value, and the variance recursive calculation also updates the new variance value based on the difference between the old and new samples. This recursive algorithm significantly reduces the computational complexity. The standardization transformation and linear scaling operation are implemented through table lookup or shift operation to avoid expensive operations such as floating-point division on resource-constrained embedded devices, thereby improving the calculation efficiency. The module has an abnormal value detection mechanism, which temporarily suspends the normalization processing and uses the last valid output when the feature value exceeds the reasonable physical range, preventing sensor failure from causing system failure.

[0031] The trainable weight parameters of the weighted combination module are stored in a non-volatile memory, and the pre-trained weight values are loaded from the memory when the system starts, and the weight values remain fixed during operation. The organization form of the weight parameters is a one-dimensional array, the array length is the same as the dimension of the standardized feature set, and the array index is one-to-one corresponding to the feature index. The multiplication operation of the feature and the weight is realized by vector dot product operation, and parallel calculation is realized by using single instruction multiple data stream instruction of embedded processor to accelerate the weighting process. The summation operation adopts accumulator structure, and each element in the weighted feature vector is traversed in sequence, and is gradually accumulated to the accumulator register, and finally the accumulated result is output as the primary fusion feature. The system is provided with a numerical overflow protection mechanism for the weighted combination process, and when the intermediate result in the accumulation process exceeds the representation range, numerical truncation or scaling is automatically performed to ensure the calculation stability. The normalization processing and the weighted combination operation of the dynamic feature aggregation module jointly constitute the feature preprocessing pipeline, the normalization processing makes the features from different sensor sources have numerical comparability, and the weighted combination selects the features based on the learned knowledge. The mean and variance of the normalization parameter are dynamically updated over time to adapt to the long-term changes of the driver state, and the weight parameter is obtained through offline training and encodes the domain knowledge. The module design considers the particularity of the mine truck driving environment, and interference factors such as vibration noise and light change may affect the stability of the feature, and the normalization processing enhances the robustness of the feature to a certain extent. The primary fusion feature after the weighted combination is used as the basis for higher-level feature fusion, and is spliced and fused with the advanced feature set to finally form a comprehensive fatigue feature representation.

[0032] In embodiment 3, the dynamic feature aggregation module splices the advanced feature set and the primary fusion feature by using vector connection, and the feature vectors in the advanced feature set and the primary fusion feature vector are connected in the feature dimension, and the connection operation arranges and combines the elements of the two vectors in sequence to form a new vector with an expanded dimension. Assuming that the advanced feature set is a column vector containing m elements, and the primary fusion feature is a scalar value, after splicing, a new feature vector with m+1 dimensions is generated. The vector connection retains all the information of the original feature without any information selection or compression, ensuring that all feature components can participate in subsequent processing. The connected feature vector is subjected to dimension reduction processing to reduce the feature dimension to a preset size, and the dimension reduction processing is realized through a fully connected neural network layer, the input dimension of the network layer is m+1, and the output dimension is a fixed value n, wherein n is much smaller than m+1. The dimension reduction process can be represented as:

[0033] Wherein: is the (m+1) -dimensional input feature vector after connection, is an n×(m+1) -dimensional projection weight matrix, is an n-dimensional bias vector, is the n-dimensional output feature vector after dimension reduction. The projection weight matrix and the bias vector are trainable parameters, which are optimized by gradient descent algorithm during model training. Dimension reduction eliminates redundant information and noise components in the feature vector, retains the most discriminative feature combinations, and reduces computational complexity and memory usage.

[0034] The hierarchical feature extraction module parses the fatigue state features from multi-source data, filters and denoises the physiological signal data. The filter processing uses a Butterworth band-pass filter, and the passband frequency range is set to 0.5Hz to 45Hz, effectively filtering out power frequency interference and baseline drift. The denoising processing uses discrete wavelet transform, and selects db4 wavelet basis function to decompose the signal for 4 layers. The detail coefficients are processed by soft threshold method to suppress random noise. The time domain and frequency domain features of the physiological signal are extracted. The time domain features include mean, variance, root mean square value and zero-crossing rate. The frequency domain features are calculated by fast Fourier transform to calculate the power spectral density, and the energy proportion of each frequency band and the center frequency are extracted. Frame sequence analysis is performed on the video image data. The frame sequence analysis uses the optical flow method to calculate the motion vector field between consecutive frames, and estimates the motion direction and speed of each pixel point through the dense optical flow algorithm. The driver's face key points and body joint points are detected. The face key point detection uses a feature point positioning model based on convolutional neural network to identify the two-dimensional coordinates of 76 key points such as eyes, eyebrows, nose and lips. The body joint point detection is realized by OpenPose pose estimation algorithm, which locates the spatial positions of 18 joint points such as head, neck, shoulder, elbow, wrist, hip, knee and ankle. The eye movement features are generated based on the face key points, including eyelid closure frequency and pupil change rate. The eyelid closure frequency is calculated by counting the number of times the eyelid distance is less than the threshold value per unit time, and the pupil change rate is calculated based on the change slope of the pupil area in consecutive frames. The head posture features are generated based on the face key points, including head rotation angle and nodding frequency. The head rotation angle is calculated by solving the rotation matrix of the head relative to the camera through PnP algorithm and converting it to Euler angle, and the nodding frequency is extracted from the periodic fluctuation frequency of the head pitch angle sequence. The body behavior features are generated based on the body joint points, including limb movement amplitude and posture change period. The limb movement amplitude is calculated by averaging the displacement distance of the joint points per unit time, and the posture change period is extracted from the joint point motion trajectory using autocorrelation analysis method.

[0035] The concatenation operation of the dynamic feature aggregation module is realized by memory block copy technology in the embedded system. The system allocates continuous storage space for the advanced feature vector and the primary fusion feature, and merges the data blocks of the two features in sequence through memory copy instruction. The full connection layer operation of dimension reduction processing is accelerated by matrix multiplication instruction. The output vector is calculated in parallel for each element using the single instruction multiple data stream architecture of the processor. The implementation of the nonlinear activation function ReLU uses conditional assignment instructions to compare each output element to zero, setting negative values to zero and leaving positive values unchanged, thereby avoiding the use of branching instructions to improve execution efficiency. The physiological signal processing flow of the hierarchical feature extraction module is deployed on a dedicated digital signal processor, and the Butterworth filter is implemented using a direct II type structure to reduce the computational complexity of the wavelet transform using the fast lifting algorithm. The video image processing flow runs on an embedded platform with GPU acceleration, and the optical flow calculation uses an optimized version of the Lucas-Kanade algorithm, and the face key point detection and body joint detection use lightweight neural network models to ensure real-time performance. The timing consistency in the feature extraction process is maintained through a frame timestamp synchronization mechanism, and the system labels each video frame and corresponding physiological signal sample with an accurate timestamp to ensure alignment of different modal features on the time axis. The layout of the feature vector in memory uses a compact arrangement to reduce cache misses and improve data access efficiency. A feature quality evaluation mechanism is provided within the module to check the reasonableness of the extracted feature values and eliminate abnormal values that are obviously outside the physiological range. The intermediate fusion feature, as an important output of the dynamic feature aggregation module, is passed to the subsequent historical pattern retrieval module to provide a multi-level fusion feature representation for fatigue state recognition. The entire processing flow is completed under strict time constraints, with an end-to-end delay from raw data input to intermediate fusion feature output controlled within 100 milliseconds, meeting the real-time requirements of mine truck driving safety warning. The allocation of computing resources of the module is dynamically adjusted according to the feature dimension, maximizing the system throughput while ensuring processing accuracy, adapting to the characteristics of long-time continuous operation. The dimension of the projection weight matrix is dynamically configured according to the input feature dimension, and the corresponding weight parameters are loaded during the system initialization stage according to the sensor configuration. The initialization of the bias vector uses a zero value initialization strategy, which is gradually adjusted to the optimal value during the training process. The dimension of the reduced feature n is set according to empirical values, usually taking one fourth to one half of the original feature dimension, balancing the amount of information and reducing the dimension. The output feature vector of the ReLU activation function will produce sparsity, with negative values set to zero and only a part of neurons activated, which helps to improve the discriminability of the feature. The intermediate fusion feature, as the input of high-level feature aggregation, will be fused with the abstract feature set through attention weighting, and finally a comprehensive fatigue feature representation will be generated. The multi-channel feature extraction pipeline of the hierarchical feature extraction module runs in parallel, and the physiological signal processing pipeline and the video image processing pipeline are accelerated through hardware accelerators to improve processing speed, and the two-channel features are aligned and merged at the time synchronization point.

[0036] Referring to Figure 4Fig. 6 is a diagram showing the trend of the three fatigue state characteristics, i.e., eyelid closure degree, head tilt angle, and body movement amplitude, with respect to working time, in the form of a multi-fold line, which is a typical output of the hierarchical feature extraction module. The eyelid closure degree belongs to the eye movement feature, the head tilt angle belongs to the head posture feature, and the body movement amplitude belongs to the body behavior feature, which correspond to different levels of fatigue state dimensions. As can be seen from the diagram, the three characteristic values as a whole show an upward trend with the passage of working time, which intuitively reflects the progressive development of the driver's fatigue state: the characteristic values grow slowly in the early stage, accelerate in the middle stage, and continue to rise with fluctuations in the later stage. The synchronous changes of these multi-dimensional features not only demonstrate the ability of the hierarchical feature extraction module to analyze multi-level fatigue features from video image data, but also provide multi-source and multi-dimensional basic inputs for the subsequent dynamic feature aggregation module. The three types of features from micro eye movement, head posture to macro body behavior comprehensively cover the multi-level performance of fatigue state, overcome the limitations of traditional single feature extraction, provide multi-modal data support for the subsequent realization of precise fatigue degree prediction and adaptive warning, and fully embody the technical advantages of the hierarchical feature extraction module in capturing subtle changes in fatigue state and constructing a comprehensive feature system.

[0037] In embodiment 4, the history pattern retrieval module calculates the similarity between the comprehensive fatigue feature representation and the stored feature representations in the history fatigue feature library, loads all the history fatigue feature representations from the history fatigue feature library, and stores the history fatigue feature representations in the form of vectors in the feature database. Each history fatigue feature representation corresponds to the feature data of a historical driving session, including the fusion results of eye movement features, head posture features and body behavior features extracted from the session. The history fatigue feature library uses a distributed database architecture to store feature vectors, and distributes a large amount of feature data on different storage nodes through sharding technology. The loading process reads the feature vectors in batches through the database query interface, and the query conditions include the time range, the driver identifier and the fatigue level label, to ensure the relevance of the retrieved feature data. The cosine similarity value between the comprehensive fatigue feature representation and each history fatigue feature representation is calculated. The cosine similarity calculates the similarity of two vectors in direction, and the value range is from negative one to positive one. The cosine similarity value is obtained by dividing the dot product of two vectors by the product of their lengths. The calculation process is not sensitive to the length of the vector, and only focuses on the relative distribution relationship in the feature space. The closer the cosine similarity value is to one, the more similar the two feature representations are, and the closer the negative one, the less similar they are. Zero value represents orthogonal independence. According to the cosine similarity value, the history fatigue feature representations are sorted. The sorting operation arranges all the similarity values in descending order using the quicksort algorithm, and the feature representation with the highest similarity is placed at the front of the list. The top-k most similar feature representations are selected, and the value of k is set as a fixed integer according to the system configuration, representing the number of nearest neighbors to be considered. The selection of k value affects the coverage and stability of the retrieval result. A larger k value makes the result more stable but increases the calculation amount, and a smaller k value is more sensitive but is easily disturbed by noise. The top-k most similar feature representations are weighted and averaged to generate a similar history fatigue pattern. The weight is calculated according to the cosine similarity value of each feature, and the higher the similarity, the greater the weight in the averaging process. The weighted average generates a vector of the same dimension as the original feature, which is output as a similar history fatigue pattern to the reinforcement learning inference module.

[0038] The historical fatigue feature library construction module collects multi-source sensor data and corresponding fatigue event labels in historical driving sessions. The multi-source sensor data includes electrocardiogram signals, electroencephalogram signals, and video image data. The data acquisition frequency is set according to the sensor type. The fatigue event labels are determined by professional annotators based on video recordings and physiological indicators. The label system uses a three-level classification: awake state, mild fatigue state, and severe fatigue state. The data of each historical driving session contains complete records within a continuous time period. The session duration varies from a few minutes to several hours, depending on the length of the specific driving task. The historical fatigue state features are extracted from the data of each historical driving session. The feature extraction process is consistent with the real-time processing module, using the same hierarchical feature extraction algorithm and parameter settings. The historical fatigue state features include eye movement features, head posture features, and body behavior features extracted from the raw data. These features are stored after standardization processing. The historical fatigue state features and fatigue event labels are stored in association in the database. The database uses a time series database structure, and each record contains feature vectors, timestamps, driver information, fatigue labels, and other metadata. The database is updated regularly to add newly collected data and labels. The update cycle is set according to the data accumulation speed, usually incremental update by day or week. Data quality checks are performed during the update process to remove invalid records and abnormal data, ensuring the reliability and consistency of the database content.

[0039] The historical pattern retrieval module uses an approximate nearest neighbor search algorithm to accelerate large-scale data retrieval. The algorithm constructs an index structure of feature vectors, dividing the high-dimensional space into multiple regions. During retrieval, only a few regions need to be searched for the nearest neighbors, avoiding full scanning of all feature vectors, significantly improving retrieval efficiency. The module has a cache mechanism to store the results of recent queries, returning cached results directly when similar queries occur again to reduce computation delay. The data processing pipeline of the historical fatigue feature library construction module uses a distributed computing framework, with multiple computing nodes processing different historical driving session data in parallel. Feature extraction tasks are distributed to different nodes for execution, and the extraction results are unified and stored in the central database. The database has an access control mechanism, only authorized modules can query and update data, protecting driver privacy information. Table 1 shows the feature representation stored in the historical fatigue feature library.

[0040] Table 1: Historical fatigue feature representation

[0041] The cosine similarity calculation of the historical pattern retrieval module is achieved through vectorization operation, and the SIMD instruction of the processor is used to calculate the similarity between multiple vectors in parallel. The top-k selection process uses a heap data structure to maintain the current maximum k similarity values, reducing the computational complexity of the sorting operation. The weighted average operation multiplies the k feature vectors with the corresponding similarity weights and sums them up, and the weights are obtained by normalizing the similarity values through the softmax function. The data collection process of the historical fatigue feature library construction module monitors the data quality in real time, fills in the missing values using interpolation methods, and detects and corrects abnormal values using statistical methods. Detailed metadata information is recorded during feature extraction, including sensor model, sampling rate, feature extraction algorithm version, etc., to ensure traceability of feature data. The database performs backup and compression operations regularly, and the backup data is stored in a remote disaster recovery center to ensure data security and system recoverability. The historical pattern retrieval module and the historical fatigue feature library construction module work together, with the retrieval module relying on the construction module to provide high-quality feature data, and the construction module optimizing the feature extraction strategy based on the feedback from the retrieval module. Through continuous learning and updating, the entire system gradually improves the accuracy and adaptability of fatigue warning, meeting the long-term monitoring needs of mine truck driving environment. The stability of the retrieval results is evaluated by the consistency of multiple retrievals, and the weighted average similarity historical fatigue pattern is used as an important input for reinforcement learning reasoning, providing historical context reference for fatigue prediction. The index structure of the database is optimized regularly, and the index strategy is adjusted according to the query mode to improve the response speed of frequent queries. The storage of feature vectors uses compressed format to reduce storage space occupation, and real-time decompression during retrieval does not affect the query efficiency.

[0042] Example 5: The reinforcement learning inference module combines the comprehensive fatigue feature representation and similar historical fatigue patterns into a state vector. The construction of the state vector involves data alignment and concatenation from two input sources. The comprehensive fatigue feature representation is a feature vector containing 64 floating-point numbers, and the similar historical fatigue pattern is another 64-dimensional vector; the two vectors are identical in dimension. The concatenation operation connects the two vectors in the first dimension, generating a 128-dimensional state vector, which serves as the input to the deep Q-network model. Each dimension of the state vector represents a specific aspect of the fatigue state, comprehensively covering driver fatigue information from basic physiological features to historical pattern features. The state vector is input into a pre-trained deep Q-network model, which is a feedforward neural network with three fully connected layers. The first layer contains 256 neurons, receiving the 128-dimensional state vector input; the second layer contains 128 neurons; and the third layer is the output layer, with the number of neurons matching the action space size. Each fully connected layer is followed by a ReLU activation function to introduce non-linear transformation capabilities, and the output layer directly outputs the Q-value using a linear activation function. The Q-value of each possible action is calculated through forward propagation using a deep Q-network model. The forward propagation process passes the state vector sequentially through three fully connected layers, with the output of each layer serving as the input to the next. The network weights and bias parameters are optimized during pre-training and remain fixed during inference. For a given state vector, the network outputs a vector containing three Q-values, corresponding to the expected cumulative reward of three selectable actions. The action with the largest Q-value is selected as the fatigue prediction strategy, outputting the driver's real-time fatigue prediction. The Q-value comparison process identifies the maximum of the three output values, and its corresponding action index determines the final fatigue level. The action space is defined as three discrete values: 0 represents alertness, 1 represents mild fatigue, and 2 represents severe fatigue. The predicted values ​​are mapped to a continuous interval from zero to one, where zero represents complete alertness and one represents extreme fatigue.

[0043] The fatigue state determination module generates a warning signal according to the comparison result of the real-time fatigue degree prediction value of the driver and the preset threshold value, and sets multiple fatigue degree threshold values including an attention threshold value and a danger threshold value. The attention threshold value is set to 0.6, corresponding to the fatigue degree that needs to be paid attention to; and the danger threshold value is set to 0.8, corresponding to the dangerous fatigue state that needs to be intervened immediately. The threshold value is determined based on a large amount of historical data analysis and field expert experience, and can be fine-tuned in different driving environments. When the real-time fatigue degree prediction value of the driver exceeds the attention threshold value but is lower than the danger threshold value, a low-level warning signal is generated, triggering a visual reminder. The visual reminder is realized by the LED indicator light in the cockpit, the indicator light flashes slowly in green, the flashing frequency is once per second, and the reminder strength is low to avoid interfering with normal driving. When the real-time fatigue degree prediction value of the driver exceeds the danger threshold value, a high-level warning signal is generated, triggering a visual and auditory reminder. The visual reminder is upgraded to a red indicator light that flashes quickly, with a frequency of three times per second, and the auditory reminder is a specific pattern of beeping sound emitted by the vehicle-mounted speaker. The reminder strength and duration are adjusted according to the warning type, and the duration of the low-level warning is fixed at five seconds, and the high-level warning lasts until the fatigue degree prediction value falls below the danger threshold value.

[0044] The implementation of the reinforcement learning inference module adopts a special neural network inference engine, which is optimized for embedded devices and supports fixed-point number operation to accelerate inference speed. The state vector splicing operation is directly performed in memory, avoiding unnecessary copy operations and reducing memory occupation and delay. The parameters of the deep Q network model are stored in the read-only memory and loaded into the cache as needed during inference, improving data access efficiency. Network calculation adopts inter-layer pipeline parallelism, with hidden layer calculation overlapping with output layer calculation to maximize hardware utilization. The threshold comparison circuit of the fatigue state determination module is realized by a digital comparator, and the prediction value input is compared with multiple threshold values in parallel, completing all threshold judgments in a single clock cycle. The visual reminder control unit generates accurate pulse width modulation signals to drive the LED indicator light, and the pulse frequency and duty cycle are dynamically adjusted according to the warning level. The auditory reminder unit integrates multiple warning audio modes, each mode corresponding to a different fatigue level, and the audio data is stored in a compressed format and decompressed in real time during playback.

[0045] The deep Q network model training phase of the reinforcement learning inference module uses historical driving data containing a large number of state vectors and corresponding optimal action labels. The training process uses an experience replay mechanism to update network parameters by randomly sampling batches of transition samples, improving training stability. The warning strategy of the fatigue state determination module can be customized according to the specific mine environment. In noisy work areas, the intensity of auditory reminders can be increased, and in environments with high visual interference, the brightness of indicator lights can be adjusted. The modules communicate through standard interfaces, and the fatigue prediction value output by the reinforcement learning inference module is transmitted to the fatigue state determination module through shared memory. The system monitors the warning trigger frequency and duration in real time, and automatically starts the diagnostic program when the number of warnings per unit time exceeds the normal range, checking whether the sensors and data flow are normal. The entire warning process is completed within strict time constraints, with a total delay from state vector input to warning trigger controlled within 200 milliseconds, ensuring timely response to fatigue states. Warning history data is regularly uploaded to the central server for analysis to optimize threshold settings and warning strategies, forming a closed-loop improvement mechanism. Feedback information from the driver is collected through the human-machine interface to evaluate the warning effect and adjust the warning method, improving the practicality and acceptance of the system.

[0046] Referring to Figure 5 The figure directly compares the total duration and occurrence frequency of the three categories of no warning, low-level warning, and high-level warning in the form of a double histogram. The figure accurately presents the hierarchical warning logic of the fatigue state determination module, reflecting the system's ability to dynamically generate warning signals of different intensities based on the comparison of the driver's real-time fatigue prediction value and the preset threshold. By quantifying the duration and frequency of different warning levels, the distribution of mine truck drivers' fatigue states is directly presented, and data support is provided for optimizing warning thresholds and adjusting warning intensity and duration, ultimately ensuring the timeliness and accuracy of fatigue warnings in mine truck driving and effectively reducing the risk of accidents caused by fatigue driving.

[0047] Although embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A driver fatigue monitoring system based on reinforcement learning, characterized in that, include: The multi-source data acquisition module is used to collect physiological signal data and video image data of mining truck drivers in real time; The hierarchical feature extraction module is used to extract multiple fatigue state features at different levels from the physiological signal data and video image data. The fatigue state features include eye movement features, head posture features, and body behavior features. The dynamic feature aggregation module is used to progressively fuse the fatigue state features at multiple different levels to generate a comprehensive fatigue feature representation. The historical pattern retrieval module is used to calculate the similarity between the comprehensive fatigue feature representation and the feature representation stored in the historical fatigue feature database, and to obtain similar historical fatigue patterns based on the similarity results. The reinforcement learning inference module is used to input the comprehensive fatigue feature representation and the similar historical fatigue pattern into the reinforcement learning model and output the driver's real-time fatigue prediction value. The fatigue state determination module is used to determine whether the driver is in a fatigue state based on the comparison result between the driver's real-time fatigue prediction value and the preset threshold.

2. The driver fatigue monitoring system based on reinforcement learning according to claim 1, characterized in that, The dynamic feature aggregation module is specifically used for: An initial feature set for the basic level is identified from the fatigue state features of the multiple different levels; The initial feature set is normalized to generate a standardized feature set; The features in the standardized feature set are weighted and combined to generate primary fusion features; Identify the intermediate-level advanced feature set from the multiple different levels of fatigue state features; The advanced feature set is concatenated with the primary fusion feature to generate the intermediate fusion feature; Identify a high-level set of abstract features from the multiple different levels of fatigue state features; The abstract feature set and the intermediate fusion feature are then fused with attention weighting to generate the comprehensive fatigue feature representation.

3. The driver fatigue monitoring system based on reinforcement learning according to claim 2, characterized in that, The dynamic feature aggregation module normalizes the initial feature set in the following ways: Calculate the mean and variance of each feature in the initial feature set; Based on the mean and variance, each feature is standardized to ensure that the feature value distribution is within a preset range. The standardized features are linearly scaled to generate the standardized feature set.

4. The driver fatigue monitoring system based on reinforcement learning according to claim 2, characterized in that, The dynamic feature aggregation module performs weighted combination of the standardized feature set in the following ways: Assign trainable weight parameters to each feature in the standardized feature set; Each feature is multiplied by its corresponding weight parameter to obtain the weighted features; The primary fusion feature is generated by summing all weighted features.

5. The driver fatigue monitoring system based on reinforcement learning according to claim 2, characterized in that, The dynamic feature aggregation module performs the concatenation operation between the advanced feature set and the primary fusion feature in the following ways: The feature vectors in the advanced feature set are connected to the primary fused feature vectors along the feature dimension; The concatenated feature vectors are then subjected to dimensionality reduction processing to reduce the feature dimension to a preset size; The dimensionality-reduced feature vector is transformed by applying a nonlinear activation function to generate the intermediate fusion feature.

6. The driver fatigue monitoring system based on reinforcement learning according to claim 1, characterized in that, The hierarchical feature extraction module is specifically used for: The physiological signal data is filtered and denoised to extract time-domain and frequency-domain features; Frame sequence analysis is performed on the video image data to detect key facial points and body joints of the driver; The eyelid closure frequency and pupil change rate are calculated based on the facial key points to generate the eye movement features; The head rotation angle and nodding frequency are calculated based on the facial key points to generate the head posture features; The body's movement amplitude and posture change cycle are calculated based on the body's joints to generate the body's behavioral characteristics.

7. The driver fatigue monitoring system based on reinforcement learning according to claim 1, characterized in that, The historical pattern retrieval module is specifically used for: Load all historical fatigue feature representations from the historical fatigue feature library; Calculate the cosine similarity value between the comprehensive fatigue feature representation and each historical fatigue feature representation; The historical fatigue feature representations are sorted according to the cosine similarity value, and the top-k most similar feature representations are selected. The top-k most similar feature representations are weighted and averaged to generate the similar historical fatigue pattern.

8. The driver fatigue monitoring system based on reinforcement learning according to claim 1, characterized in that, It also includes a historical fatigue feature library building module, used for: Collect multi-source sensor data and corresponding fatigue event labels from historical driving sessions; Feature extraction is performed on the data from each historical driving session to generate historical fatigue state features; The historical fatigue state characteristics are associated with fatigue event tags and stored in the database; the database is updated regularly to add newly collected data and tags.

9. The driver fatigue monitoring system based on reinforcement learning according to claim 1, characterized in that, The reinforcement learning inference module is specifically used for: The comprehensive fatigue feature representation and the similar historical fatigue patterns are combined into a state vector; The state vector is input into a pre-trained deep Q-network model; The Q-value of each possible action is calculated through forward propagation using a deep Q-network model; The action with the largest Q value is selected as the fatigue prediction strategy, and the real-time fatigue prediction value of the driver is output.

10. The driver fatigue monitoring system based on reinforcement learning according to claim 1, characterized in that, The fatigue state determination module is specifically used for: Set multiple fatigue thresholds, including a attention threshold and a danger threshold; When the driver's real-time fatigue prediction value exceeds the attention threshold but is below the danger threshold, the driver is determined to be in a state of low fatigue, and a low-level warning signal is generated to trigger a visual reminder. When the driver's real-time fatigue prediction value exceeds the danger threshold, the driver is determined to be in a state of high fatigue, and an advanced warning signal is generated. At the same time, visual and auditory reminders are triggered, and the intensity and duration of the reminders are adjusted according to the type of warning.

Images