Locomotive driver fatigue state detection method

Abstract

The invention relates to the technical field of computers, and provides a locomotive driver fatigue state detection method, which comprises the following steps of: S1, establishing a user image sample data set of a locomotive driver fatigue state and marking; s2, inputting a generative adversarial network model, and training a generator and a discriminator of the generative adversarial network model to perform data expansion; s3, after data expansion, inputting the data into a feature extraction model, a feature dimension reduction model and a classifier model for model training to obtain a trained user image sample model; and S4, based on the trained user image sample model, monitoring the fatigue level of the locomotive driver in real time, and giving out safety measures. According to the scheme, the training data set is effectively expanded based on the generative adversarial network, the training efficiency of the feature extraction model is remarkably improved, the convergence speed of the training process is accelerated, the model development period is shortened, the data reconstruction quality is improved through optimization of the feature dimension reduction model, and the classification precision is improved through improvement of the classifier.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method for detecting a fatigue state of a locomotive driver. Background Art

[0002] Motorcycle drivers may experience fatigue during long hours of driving. If this fatigue state is not identified and handled in time, it may lead to serious safety accidents. Traditional fatigue detection methods rely on subjective feedback from drivers or simpler physiological indicator monitoring. These methods are not only slow to respond, but also easily affected by individual differences, making it difficult to accurately assess the real-time fatigue state of drivers.

[0003] The Chinese invention patent with application number CN202111174577.0 proposes a fatigue driving detection method based on multi-source visual information fusion, which includes: establishing a user image sample data set under fatigue driving state; based on the user image sample data set, extracting and fusing multi-dimensional facial action units and head posture features, multi-dimensional eye texture features and multi-dimensional forehead texture features to obtain a multi-dimensional fatigue feature vector; using the MI-FCBF algorithm to select the multi-dimensional fatigue feature vector to obtain a low-dimensional fatigue feature; according to the low-dimensional fatigue feature, training the Adaboost classifier to generate a classification model that can be used to identify fatigue driving state; monitoring the user's driving state according to the classification model. The above-mentioned fatigue driving detection method based on multi-source visual information fusion fuses the facial visual features under visible light and the facial infrared thermal image texture information at the feature layer, which can comprehensively evaluate the fatigue state with visual signals and physiological signals, and the detection result is relatively objective.

[0004] The inventor of this application believes that the prior art has the following problems: (1) In terms of training data collection and utilization, common monitoring system data is insufficient to support effective model training, resulting in poor model generalization ability.

[0005] (2) The existing neural network training process has the problem of slow convergence, which affects the training efficiency.

[0006] (3) Existing feature extraction methods often lead to significant information loss when processing high-dimensional data, affecting the detection accuracy and reliability of the model.

[0007] (4) Existing classifiers are prone to the vanishing gradient problem when processing complex data, which reduces the classification effect and stability of the model under complex conditions.

[0008] Based on this, it is particularly important to develop an efficient, automated method that can monitor and evaluate driver fatigue status in real time. Summary of the invention

[0009] In view of this, based on the problems that the existing technology has low accuracy in detecting the fatigue status of locomotive drivers, it is difficult to form a large-scale training set for locomotive driver fatigue status scenarios, and there is a large deviation between images captured at different locations or devices, which makes it difficult to directly use them for model training. The present invention proposes a method, device, equipment and medium for detecting the fatigue status of a locomotive driver.

[0010] Based on the above purpose, one aspect of the present invention provides a method for detecting a fatigue state of a locomotive driver, comprising the following steps: S1. Establish a user image sample dataset of locomotive driver fatigue status and mark the fatigue level; S2, inputting the user image sample dataset into the generative adversarial network model, training the generator and discriminator of the generative adversarial network model for data expansion; S3, input the user image sample data set after data expansion into the feature extraction model, feature dimensionality reduction model and classifier model respectively, perform feature extraction model training based on a fully connected neural network, perform feature dimensionality reduction model training based on an autoencoder algorithm with regularization loss, and perform classifier model training based on a high-order neural network algorithm, to obtain a trained user image sample model; S4. Based on the trained user image sample model, the fatigue level of the locomotive driver is monitored in real time, and corresponding safety measures are given; Among them, training the generator and discriminator of the generative adversarial network model includes adjusting the parameters of the generator and the discriminator using the gradient descent method, and updating the parameters of the generator and the discriminator in combination with a dynamic adjustment function; The training focus of the generator is dynamically adjusted based on the quality of the user image sample data set and the discriminator's discrimination results. A dynamic optimization strategy is adopted to dynamically adjust the training focus of the generator based on the feedback from the discriminator. After each training cycle, the generated images are evaluated at the feature level, and the loss function is adjusted according to the feedback from the fine-grained feature module, giving priority to improving areas where features are insufficiently expressed.

[0011] In some embodiments, the parameters of the generator and the discriminator are expressed as:

[0012]

[0013] In the formula, is the loss function of the generator, is the loss function of the discriminator, is the learning rate of adversarial training, For the The parameters of the generator for the iterations, For the The parameters of the discriminator for the iteration, For the The parameters of the generator for the iterations, For the The parameters of the discriminator for the iteration, is the gradient of the generator’s loss function with respect to the generator’s parameters, is the gradient of the discriminator's loss function with respect to the discriminator parameters, is a dynamic adjustment function, is the gradient of the composite loss function with respect to the generator parameters, is the composite loss function, is the gradient of the composite loss function with respect to the discriminator parameters.

[0014] In some embodiments, the calculation method of the dynamic adjustment function is expressed as:

[0015] In the formula, represents the generated image data, is the feature recalibration factor, which represents the adjustment intensity of dynamic adjustment; is the symbol of partial derivative; The dynamic adjustment function uses the feature recalibration factor to influence the discriminator's focus, prompting the generator to focus on features that are highly correlated with fatigue status when generating new images. The calculation method of the conditional probability distribution of generated image and real image features is expressed as:

[0016] In the formula, represents the real image data, yes and The probability of the intersection of the two is is the marginal probability of the real image feature, Represents the conditional probability distribution of generated image and real image features; The feature recalibration factor is calculated based on the conditional probability distribution of the generated image and the real image features. The calculation method is expressed as:

[0017] In the formula, is a scale parameter used to adjust the effect of recalibration.

[0018] In some embodiments, the feature enhancement function is calculated as:

[0019] In the formula, is the feature enhancement function, represents the real image data, is the first Features, is the real image Features, It is The feature weight of each feature, is the similarity calculation function, is the feature enhancement coefficient; is the number of features of the image, that is, the number of pixels; The calculation method of the similarity calculation function is expressed as:

[0020] In the formula, Represents the difference between the generated image and the real image in The square of the Euclidean distance of feature dimensions, It is the L2 norm, and its calculation method is the same as that of the Euclidean distance; It is The scale parameter of each feature is a training parameter, which is adjusted by the gradient descent method to adjust the contribution of different features to the overall similarity calculation; Represents an exponential function with a natural constant as its base.

[0021] In some embodiments, the feature enhancement function is characterized as a pixel-level loss, and the composite loss function is adjusted by combining the pixel-level loss and the feature-level loss. The calculation method of the composite loss function is expressed as:

[0022] In the formula, represents the feature-level loss, is the composite loss balance coefficient; The calculation method of feature level loss is expressed as:

[0023] In the formula, and are the generated and real images, respectively. The sample Features The number of samples entered for the current batch; is a normalization constant to ensure comparability of losses between different features.

[0024] In some embodiments, the feature extraction model training based on the fully connected neural network in step S3 is specifically: S301, initialize the parameters of the neural network, and set the initial adjustment frequency and adjustment amplitude, expressed as:

[0025]

[0026] In the formula, To adjust the frequency, To adjust the initial value of the frequency, To adjust the amplitude, To adjust the initial value of the amplitude, It is a hyperparameter that is adaptively adjusted according to the network depth. is a hyperparameter that is adaptively adjusted according to the network width. To adjust the weight factor of the amplitude, is the hyperbolic tangent function, Assignment parameters to adjust the amplitude; S302, input samples and perform forward propagation, calculate the error between the network output and the actual output, expressed as:

[0027]

[0028] In the formula, is the Sigmoid activation function; is the output of the neural network, which is obtained by calculating the output of the last layer of neurons in the neural network through Softmax; is the weight of the neural network, is the bias of the neural network, is the error of the neural network, is the true label of the sample; S303, dynamically calculate the resonance factor based on the error size for subsequent frequency and amplitude adjustment, and the calculation method is expressed as:

[0029] In the formula, is the resonance factor, is a hyperparameter of the sensitivity of the reaction error to the resonance adjustment; S304, update the adjustment frequency according to the resonance factor adjustment parameter, so that it increases when the error is large and decreases when the error is small. The calculation method is expressed as:

[0030] In the formula, For the The adjustment frequency of the iterations, For the The adjustment frequency of the iterations, is the frequency adjustment rate; S305, update the size of the adjustment amplitude to ensure that the update step size matches the error and avoid oscillation caused by too large a step size. The calculation method of the adjustment amplitude is expressed as:

[0031] In the formula, For the The adjustment amplitude of the iteration, For the The adjustment amplitude of the iteration, is the adjustment angle of the amplitude; S306. Calculate and adjust the phase using the phase correction mechanism to ensure synchronization of parameter updates and optimize the convergence path. The adjustment method is expressed as:

[0032] In the formula, For the The adjustment phase of the iteration, For the The adjustment phase of the iteration, is the phase update rate, Indicates the current iteration number; S307, updating the weights and biases of the neural network according to the adjustment frequency, the adjustment amplitude and the adjustment phase, the update method is expressed as:

[0033]

[0034] In the formula, For the The weights of the neural network at the iteration, For the The bias of the neural network at the iteration, For the The weights of the neural network at the iteration, For the The bias of the neural network at the iteration, is the gradient of the neural network error with respect to the weight parameters, is the gradient of the neural network error with respect to the bias parameter; S308, repeat the above steps until the preset stop iteration condition is met, which means that the model training is completed.

[0035] In some embodiments, the importance score of the output feature of the activation function of the autoencoder is calculated as:

[0036] In the decoder stage, the features from different encoding layers are used to reconstruct the data and calculate the reconstruction error. During the training process, the loss function of the autoencoder is calculated as follows:

[0037] In the formula, is the loss function of the autoencoder; The number of samples entered for the current batch; is the first input to the autoencoder samples; After reconstruction samples; is the number of layers of the autoencoder, is the L2 norm, A regularization loss term that represents the autoencoder weights; is the regularization loss weight of the autoencoder.

[0038] In some embodiments, the importance of neurons is determined by statistically analyzing the sparsity of activation values. The calculation method of the sparsity measure of layer neurons is expressed as:

[0039] In the formula, For the autoencoder Sparsity measure of layer neurons; For the The sample in The activation value of the layer; is the sparsity threshold of the autoencoder.

[0040] In some embodiments, performing classifier model training based on a high-order neural network algorithm includes: In the output layer of the high-order neural network, the Softmax function is used to calculate the predicted probability of each category, and the error between the true label and the predicted probability is calculated by combining the adaptive regularization term with the cross entropy loss function. The calculation method of the loss function is expressed as:

[0041] In the formula, It is One-hot encoding of the true labels; The model predicts The probability of each category; is the total number of categories; is the total number of layers of the high-order neural network; For high-order neural networks Adaptive regularization term for iterations; is the loss function of the high-order neural network; is the number of iterations up to the current time; The adaptive regularization term dynamically adjusts the loss function according to the state of the network at different training stages. This mechanism can reduce the risk of overfitting in the early training stage and maintain the robustness of the model in the later training. Specifically, the calculation method of the adaptive regularization term is expressed as:

[0042] In the formula, is the initial adaptive regularization term coefficient, is the parameter for the adaptive regularization term, For the The squared norm of the layer weight updates, is the total number of layers of the high-order neural network, A small constant to avoid division by zero.

[0043] Furthermore, the model predicts The calculation method of the probability of each category is expressed as:

[0044] In the formula, is the output of the last layer of the network.

[0045] In some embodiments, a gradient correction strategy is used to calculate the parameter increment of the high-order neural network, which is expressed as:

[0046] In the formula, is the learning rate of the high-order neural network, is the gradient of the loss function of the high-order neural network with respect to the weight parameters, is the element-wise multiplication operation, is a symbolic function; is the maximum value function; Accumulate impact factors for gradients to smooth the update process; is the weight of the previous layer of high-order neural network.

[0047] The present invention has at least the following beneficial effects: (1) by using a generative adversarial network, the diversity and quantity of training samples are enhanced, so that the model can be better generalized to unseen data, thereby improving the accuracy of fatigue detection; (2) the training efficiency of the feature extraction model is significantly improved, the convergence speed during the training process is accelerated, and the model development cycle is shortened; (3) the optimization of the feature dimensionality reduction model reduces the information loss of the original data and improves the quality of data reconstruction, thereby enhancing the expression ability of the fatigue state and the accuracy of detection; (4) the classifier model training improves the classification accuracy, especially the performance is more stable in complex situations, effectively reducing the risk of misclassification.

[0048] In addition, the locomotive driver fatigue state detection device, electronic equipment and medium of the present invention also have at least the above beneficial effects, which will not be repeated. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying creative work.

[0050] Figure 1 A flow chart showing a method for detecting a fatigue state of a locomotive driver according to an exemplary embodiment of the present invention is shown; Figure 2 A schematic diagram showing a portion of an image of a Yawdd dataset according to an exemplary embodiment of the present invention is shown; Figure 3 A structural diagram of a generative adversarial network model according to an exemplary embodiment of the present invention is shown; Figure 4 The training process of the autoencoder algorithm based on regularization loss according to an exemplary embodiment of the present invention is shown; Figure 5 A schematic structural diagram of a fatigue state detection device according to an exemplary embodiment of the present invention is shown; Figure 6 A schematic structural diagram of an electronic device according to an exemplary embodiment of the present invention is shown; Figure 7 A schematic structural diagram of a computer-readable storage medium according to an exemplary embodiment of the present invention is shown. DETAILED DESCRIPTION

[0051] The detailed description of the following embodiments is used to exemplify the principles of the present invention, but cannot be used to limit the scope of the present invention. The present invention can be implemented in many different forms and is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

[0052] The present invention provides these embodiments to make this disclosure thorough and complete, and fully express the scope of the present invention to those skilled in the art. It should be noted that unless otherwise specifically stated, the relative arrangement of the parts and steps, the composition of materials, numerical expressions and numerical values ​​set forth in these embodiments should be interpreted as being merely exemplary, rather than as limiting.

[0053] It should be noted that, in the description of the present invention, unless otherwise specified, the meaning of "multiple" is greater than or equal to two; the terms "upper", "lower", "left", "right", "inner", "outer", etc., indicating the orientation or positional relationship, are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0054] It should also be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. When a specific device is described as being located between a first device and a second device, there may or may not be an intermediate device between the specific device and the first device or the second device.

[0055] All terms used in the present invention have the same meanings as those understood by those of ordinary skill in the art to which the present disclosure belongs, unless otherwise specifically defined. It should also be understood that terms defined in general dictionaries, for example, should be interpreted as having meanings consistent with their meanings in the context of the relevant technology, and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined herein.

[0056] Technologies, methods, and apparatus known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, these technologies, methods, and apparatus should be considered part of the specification.

[0057] One or more embodiments of the present application will be described below with reference to the accompanying drawings.

[0058] Based on the above objectives, a first aspect of an embodiment of the present invention provides an embodiment of a method for detecting a fatigue state of a locomotive driver. Figure 1 FIG. 1 is a flow chart of a method for detecting a fatigue state of a locomotive driver provided by an embodiment of the present invention. Figure 1 As shown, including: S1. Establish a user image sample dataset of locomotive driver fatigue status and mark the fatigue level; S2, inputting the user image sample dataset into the generative adversarial network model, training the generator and discriminator of the generative adversarial network model for data expansion; S3, input the user image sample data set after data expansion into the feature extraction model, feature dimensionality reduction model and classifier model respectively, perform feature extraction model training based on a fully connected neural network, perform feature dimensionality reduction model training based on an autoencoder algorithm with regularization loss, and perform classifier model training based on a high-order neural network algorithm, to obtain a trained user image sample model; S4. Based on the trained user image sample model, the fatigue level of the locomotive driver is monitored in real time, and corresponding safety measures are given; Among them, training the generator and discriminator of the generative adversarial network model includes adjusting the parameters of the generator and the discriminator using the gradient descent method, and updating the parameters of the generator and the discriminator in combination with a dynamic adjustment function; The training focus of the generator is dynamically adjusted based on the quality of the user image sample data set and the discriminator's discrimination results. A dynamic optimization strategy is adopted to dynamically adjust the training focus of the generator based on the feedback from the discriminator. After each training cycle, the generated images are evaluated at the feature level, and the loss function is adjusted according to the feedback from the fine-grained feature module, giving priority to improving areas where features are insufficiently expressed.

[0059] The above-mentioned method for detecting the fatigue state of locomotive drivers enhances the diversity and quantity of training samples by using a generative adversarial network, so that the model can better generalize to unseen data and improve the accuracy of fatigue detection; the training efficiency of the feature extraction model is significantly improved, the convergence speed in the training process is accelerated, and the model development cycle is shortened; the optimization of the feature dimensionality reduction model reduces the information loss of the original data and improves the data reconstruction quality, thereby enhancing the expression ability of the fatigue state and the accuracy of detection; the classifier model training improves the classification accuracy, especially the performance is more stable in complex situations, effectively reducing the risk of misclassification.

[0060] The acquisition, labeling and preprocessing of training data are time-consuming and labor-intensive, and insufficient training samples can easily lead to poor generalization ability of the model, while affecting the accuracy of the model. The present invention uses a generative adversarial network based on fine-grained feature enhancement to generate samples, thereby achieving data expansion. The structure of the generative adversarial network includes two parts: a generator and a discriminator. The generator is responsible for generating new image data that is sufficient to mislead the discriminator, while the discriminator attempts to distinguish between real data and false data generated by the generator. Based on the traditional generative adversarial network, the present invention dynamically adjusts the training focus of the generator and the discriminator according to the data features in the training process, and optimizes the direction and quality of data generation. In addition, the present invention adopts a fine-grained feature enhancement strategy, which can capture key fatigue features such as faces and eyes, and ensure the authenticity and detail performance of the generated images in terms of fatigue features.

[0061] Specifically, the present invention uses a 6-layer fully connected neural network for feature extraction. In the prior art, some schemes use neural networks for feature extraction. In some neural network structures, gradient vanishing, gradient explosion or falling into the local optimal solution may be encountered, affecting the stability of training and the performance of the model. The present invention adopts an adaptive coherent optimization algorithm, which is inspired by the resonance principle in physics, that is, the system will resonate at a certain or certain specific frequencies. The adaptive coherent optimization algorithm simulates this phenomenon, and adjusts the update frequency and amplitude of the neural network parameters so that the network reaches parameter resonance during the training process, thereby quickly converging to the optimal solution. Traditional algorithms such as gradient descent use a fixed learning rate, while the parameter update frequency in the adaptive coherent optimization algorithm is an adaptive adjustment method, which is dynamically adjusted according to error feedback to meet the needs of different training stages.

[0062] The present invention adopts an autoencoder algorithm based on regularization loss to perform feature dimensionality reduction. The autoencoder consists of a symmetrical encoder and a decoder. The present invention adopts a dynamic routing mechanism based on feature importance scoring to automatically determine the transmission path of each feature in different layers of the encoder, thereby enhancing the model's ability to reconstruct the original data and reducing information loss.

[0063] The reduced-dimensional data is input into the classifier to train the classifier model. The present invention adopts a high-order neural network algorithm as the classifier model. On the basis of the traditional high-order neural network, in order to overcome the gradient vanishing problem in training, the present invention dynamically adjusts the flow direction and size of the gradient during each back propagation to ensure that the gradient can be effectively transmitted to all network layers, thereby improving the classification accuracy.

[0064] According to some embodiments of the present invention, the user image sample data set includes the eye state and / or facial state and / or body state of the locomotive driver in the driving state. The data collection involved in the present invention mainly comes from the monitoring screen of the locomotive driver. The data of these monitoring screens capture various states of the driver during operation in the form of images, including but not limited to facial expressions, eye states and overall postures. The collected data is stored in the form of jpg files.

[0065] According to some embodiments of the present invention, in step S2, training the generator and the discriminator of the generative adversarial network model for data expansion includes using image data in an open source dataset as training samples for training. The open source dataset includes the Yawdd dataset.

[0066] The Yawdd dataset contains hundreds of video clips, which were recorded by cameras in the car and simulated by dozens of participants driving the vehicle in a real driving environment. The video mainly captures the upper body of the driver, and clearly simulates the different state characteristics of the driver during driving (including talking state, natural state, fatigue state, etc.), and the video information collected by the dataset is also very comprehensive, that is, it covers gender differences and vision differences. In addition, the dataset also captures the driver's facial state from two angles. One part of the video is taken from the left front of the driver (mainly the area below the front mirror in the car), and the other part is taken from the camera directly in front of the driver (mainly the position directly above the steering wheel). Some images in the video clips of the dataset are screenshotted and used as part of the training data.

[0067] According to some embodiments of the present invention, in step S1, marking the fatigue level includes presetting the fatigue level and marking the images in the user image sample dataset.

[0068] Specifically, in the present invention, the data is annotated manually or automatically, and each image is annotated with a fatigue level to ensure the accuracy of subsequent model training. The specific annotation categories are as follows: Level 0: The driver is in good condition and shows no signs of fatigue; Level 1: The driver is slightly fatigued; Level 2: The driver is moderately fatigued; Level 3: The driver is severely fatigued.

[0069] Among them, in the case of manual labeling, the labeling personnel will classify and label according to the driver's fatigue status.

[0070] In the case of automatic labeling, the camera is used to classify and label the drivers according to their fatigue status.

[0071] When the eye condition, facial condition or body condition is identified to be good, it can be judged as level 0; When slight fatigue of the eyes, face or body is detected, it can be judged as level 1; When moderate fatigue of the eye, face or body is detected, it can be judged as level 2; When severe fatigue of the eyes, face or body is detected, it can be judged as Level 3.

[0072] According to some embodiments of the present invention, classifier model training includes dynamically adjusting the direction and size of the classifier parameter gradient during each back propagation to ensure that the gradient can be effectively transmitted to all network layers to improve classification accuracy.

[0073] According to some embodiments of the present invention, once a fatigue state is detected, the system will automatically trigger a safety warning, such as an audible alarm or a vibrating seat, to wake up the driver or notify a control center to take further action.

[0074] In order to further understand the method for detecting the fatigue state of a locomotive driver of the present invention, it is described in detail in a specific example below.

[0075] S1. Data collection and annotation The data collection involved in the present invention mainly comes from the monitoring screen of the locomotive driver. These monitoring data capture various states of the driver during operation in the form of images, including but not limited to facial expressions, eye states and overall posture, etc. The collected data is stored in jpg files.

[0076] In addition, the present invention also uses image data in open source datasets as training samples, such as Figure 2 As shown in the figure, the open source dataset is specifically the Yawdd dataset, which contains hundreds of video clips, which are recorded by the camera inside the car and obtained by dozens of participants simulating driving a vehicle in a real driving environment. The video mainly captures the upper body of the driver, and clearly simulates the different state characteristics of the driver in driving (including speaking state, natural state, fatigue state, etc.), and the video information collected by the dataset is also very comprehensive, that is, it covers gender differences, vision differences, etc. In addition, the dataset also captures the driver's facial state from two angles. One part of the video is taken from the left front of the driver (mainly the area below the front mirror inside the car), and the other part is obtained from the camera directly in front of the driver (mainly the position directly above the steering wheel). After screenshots of some images in the video clips of the dataset, they are used as part of the training data.

[0077] Specifically, in the present invention, the data is annotated manually or automatically, and each image is annotated with a fatigue level to ensure the accuracy of subsequent model training. The specific annotation categories are as follows: Level 0: The driver is in good condition and shows no signs of fatigue; Level 1: The driver is slightly fatigued; Level 2: The driver is moderately fatigued; Level 3: The driver is severely fatigued.

[0078] Among them, in the case of manual labeling, the labeling personnel will classify and label according to the driver's fatigue status.

[0079] In the case of automatic labeling, the camera is used to classify and label the drivers according to their fatigue status.

[0080] When the eye condition, facial condition or body condition is identified to be good, it can be judged as level 0; When slight fatigue of the eyes, face or body is detected, it can be judged as level 1; When moderate fatigue of the eye, face or body is detected, it can be judged as level 2; When severe fatigue of the eyes, face or body is detected, it can be judged as Level 3.

[0081] S2. Data expansion It is understandable that in the task of the present invention, the collection, acquisition, labeling and preprocessing of training data are time-consuming and labor-intensive, and insufficient training samples can easily lead to poor generalization ability of the model and affect the accuracy of the model. The present invention adopts a generative adversarial network model based on fine-grained feature enhancement to generate samples, thereby achieving data expansion. The structure of the generative adversarial network model includes two parts: a generator and a discriminator, such as Figure 3 As shown. The generator is responsible for generating new image data that is sufficient to mislead the discriminator, while the discriminator attempts to distinguish between real data and false data generated by the generator. Based on the traditional generative adversarial network, the present invention dynamically adjusts the training focus of the generator and the discriminator according to the data features in the training process, and optimizes the direction and quality of data generation. In addition, the present invention adopts a fine-grained feature enhancement strategy, which can capture key fatigue features such as faces and eyes, and ensure the authenticity and detailed expression of fatigue features in the generated images.

[0082] Specifically, the training process of the generative adversarial network algorithm based on fine-grained feature enhancement is as follows: S201, initialize the network parameters of the generator and the discriminator. The initialization method is expressed as:

[0083]

[0084] In the formula, represents a generator, represents the discriminator, represents the generator parameters, represents the discriminator parameters, are the parameters of the generator at the beginning of training, are the parameters of the discriminator at the initial stage of training; is a random initialization function, and the initialization obeys a random distribution in the interval [0,0.1].

[0085] S202. In the adversarial training stage, the parameters of the generator and the discriminator are adjusted using the gradient descent method, and the parameters of the generator and the discriminator are updated in combination with the dynamic adjustment function, which is expressed as:

[0086]

[0087] In the formula, is the loss function of the generator, is the loss function of the discriminator, is the learning rate of adversarial training, For the The parameters of the generator for the iterations, For the The parameters of the discriminator for the iteration, For the The parameters of the generator for the iterations, For the The parameters of the discriminator for the iteration, is the gradient of the generator’s loss function with respect to the generator’s parameters, is the gradient of the discriminator's loss function with respect to the discriminator parameters, is a dynamic adjustment function, is the gradient of the composite loss function with respect to the generator parameters, is the composite loss function, is the gradient of the composite loss function with respect to the discriminator parameters. Preferably, Set to 0.01.

[0088] S203, by monitoring the quality of the generated image and the discrimination result of the discriminator, the training focus of the generator is dynamically adjusted. The present invention adopts a dynamic optimization strategy to dynamically adjust the training focus of the generator according to the feedback of the discriminator. The calculation method of the dynamic adjustment function is expressed as:

[0089] In the formula, represents the generated image data, is the feature recalibration factor, which represents the adjustment intensity of dynamic adjustment; is the sign of the partial derivative. Preferably, Set to 0.1.

[0090] In one embodiment, the dynamic adjustment function uses the feature recalibration factor to influence the discriminant focus of the discriminator, so that the generator can focus more on features that are highly related to the fatigue state when generating new images, such as subtle eye movements, thereby improving the quality of the generated image and the accuracy of fatigue detection. Specifically, the conditional probability distribution of the generated image and the real image features is calculated, and the calculation method is expressed as:

[0091] In the formula, represents the real image data, yes and The probability of the intersection of the two is is the marginal probability of the real image feature, Represents the conditional probability distribution of generated image and real image features.

[0092] Furthermore, the feature recalibration factor is calculated according to the conditional probability distribution of the generated image and the real image features. The calculation method is expressed as:

[0093] In the formula, is a scale parameter used to adjust the recalibration effect. Preferably, Set to 0.3.

[0094] S204. After each training cycle, the generated image is evaluated at the feature level, and the loss function is adjusted according to the feedback of the fine-grained feature module, and those areas with insufficient feature expression are preferentially improved. The calculation method of the feature enhancement function is expressed as:

[0095] In the formula, is the feature enhancement function, represents the real image data, is the first Features, is the real image Features, It is The feature weight of each feature, is the similarity calculation function, is the feature enhancement coefficient; is the number of image features, i.e. the number of pixels. Preferably, Set to 0.2.

[0096] In one embodiment, the calculation method of the similarity calculation function is expressed as:

[0097] In the formula, Represents the difference between the generated image and the real image in The square of the Euclidean distance of feature dimensions, It is the L2 norm, and its calculation method is the same as that of the Euclidean distance; It is The scale parameter of each feature is a training parameter, which is adjusted by the gradient descent method to adjust the contribution of different features to the overall similarity calculation; Represents an exponential function with a natural constant as its base.

[0098] S205, the feature enhancement function is characterized as a pixel-level loss, and the pixel-level loss and the feature-level loss are combined to adjust the loss function to further refine the quality of the generated image, especially in the subtle expressions and dynamics of simulating fatigue. The calculation method of the composite loss function is expressed as:

[0099] In the formula, represents the feature-level loss, is the composite loss balance coefficient. Preferably, Set to 0.3.

[0100] In one embodiment, the feature level loss is calculated as:

[0101] In the formula, and are the generated and real images, respectively. The sample Features The number of samples entered for the current batch; is a normalization constant to ensure comparability of losses between different features. Preferably, Set to 10.

[0102] S206, repeat the above steps until the preset stop iteration condition is met, which means that the model training is completed. In one embodiment, the preset stop iteration condition is reaching a preset maximum number of iterations, preferably, the preset maximum number of iterations is set to 1000 times.

[0103] After the data expansion model training is completed, the trained data expansion model is used to increase the number of samples. In one embodiment, assuming that the original collected samples are 800, and the data expansion model expands and generates 200 samples, the expanded data set contains 1000 samples.

[0104] S3. Feature extraction model training The expanded data is input into the feature extraction model to train the feature extraction model. The present invention uses a 6-layer fully connected neural network for feature extraction, and uses an adaptive coherent optimization algorithm. Inspired by the resonance principle in physics, that is, the system will resonate at a certain or certain specific frequencies. The adaptive coherent optimization algorithm simulates this phenomenon, and adjusts the update frequency and amplitude of the neural network parameters so that the network reaches parameter resonance during the training process, thereby quickly converging to the optimal solution. Traditional algorithms such as gradient descent use a fixed learning rate, while the parameter update frequency in the adaptive coherent optimization algorithm is an adaptive adjustment method, which is dynamically adjusted according to error feedback to meet the needs of different training stages.

[0105] Specifically, the process of the neural network algorithm based on adaptive homology optimization is as follows: S301, initialize the parameters of the neural network, and set the initial adjustment frequency and adjustment amplitude, expressed as:

[0106]

[0107] In the formula, To adjust the frequency, To adjust the initial value of the frequency, To adjust the amplitude, To adjust the initial value of the amplitude, It is a hyperparameter that is adaptively adjusted according to the network depth. is a hyperparameter that is adaptively adjusted according to the network width. To adjust the weight factor of the amplitude, is the hyperbolic tangent function, is a parameter for adjusting the amplitude. Set to 0.5.

[0108] In one embodiment, the function of the complexity of the network structure characterized by the hyperparameters adaptively adjusted according to the network depth is calculated as follows:

[0109] In the formula, represents the total number of nodes in the neural network, Represents the number of layers of the neural network.

[0110] Moreover, the assignment parameter of the adjustment amplitude is calculated based on the initial sensitivity of the network and is expressed as:

[0111] In the formula, represents the initial weight vector elements, is the length of the weight vector.

[0112] Furthermore, let the adjustment phase be , adjust the initial value of the phase to , in this embodiment, Set to .

[0113] S302, input samples and perform forward propagation, calculate the error between the network output and the actual output, expressed as:

[0114]

[0115] In the formula, is the Sigmoid activation function; is the output of the neural network, which is obtained by calculating the output of the last layer of neurons in the neural network through Softmax; is the weight of the neural network, is the bias of the neural network, is the error of the neural network, is the true label of the sample.

[0116] S303, dynamically calculate the resonance factor based on the error size for subsequent frequency and amplitude adjustment, and the calculation method is expressed as:

[0117] In the formula, is the resonance factor, is a hyperparameter of the sensitivity of the reaction error to the resonance adjustment.

[0118] In one embodiment, the hyperparameter of the sensitivity of the response error to the resonance adjustment represents the degree of nonlinear adjustment of the error response, and the calculation method is expressed as:

[0119] In the formula, is the resonance adjustment sensitivity influencing factor. Preferably, Set to 0.005.

[0120] S304, update the adjustment frequency according to the resonance factor adjustment parameter, so that it increases when the error is large and decreases when the error is small. The calculation method is expressed as:

[0121] In the formula, For the The adjustment frequency of the iterations, For the The adjustment frequency of the iterations, is the frequency adjustment rate.

[0122] In one embodiment, the frequency adjustment rate is calculated as:

[0123] In the formula, To obtain the minimum function, is the error sensitivity coefficient, It is The average error of the iterations before the iteration. Preferably, Set to 0.1.

[0124] S305, update the size of the adjustment amplitude to ensure that the update step size matches the error and avoid oscillation caused by too large a step size. The calculation method of the adjustment amplitude is expressed as:

[0125] In the formula, For the The adjustment amplitude of the iteration, For the The adjustment amplitude of the iteration, is the adjustment angle of the amplitude.

[0126] In one embodiment, the amplitude adjustment angle is calculated as follows:

[0127] In the formula, For the The maximum error of the iteration before the iteration, For the The minimum error of the iteration before iteration .

[0128] S306. Calculate and adjust the phase using the phase correction mechanism to ensure synchronization of parameter updates and optimize the convergence path. The adjustment method is expressed as:

[0129] In the formula, For the The adjustment phase of the iteration, For the The adjustment phase of the iteration, is the phase update rate, Indicates the current iteration number.

[0130] In one embodiment, the update rate of the phase is adjusted as follows:

[0131] In the formula, represents the variance of the error, Represents the standard deviation of the error.

[0132] S307, updating the weights and biases of the neural network according to the adjusted frequency, adjusted amplitude and adjusted phase, the updating method is expressed as:

[0133]

[0134] In the formula, For the The weights of the neural network at the iteration, For the The bias of the neural network at the iteration, For the The weights of the neural network at the iteration, For the The bias of the neural network at the iteration, is the gradient of the neural network error with respect to the weight parameters, is the gradient of the neural network error with respect to the bias parameter.

[0135] S308, repeat the above steps until the preset stop iteration condition is met, which means that the model training is completed. In one embodiment, the preset stop iteration condition is reaching a preset maximum number of iterations, preferably, the preset maximum number of iterations is set to 1000 times.

[0136] S4. Feature dimensionality reduction model training The data after feature extraction is input into the feature dimensionality reduction model to train the feature dimensionality reduction model. The present invention adopts an autoencoder algorithm based on regularization loss to perform feature dimensionality reduction. The autoencoder consists of a symmetric encoder and decoder. The present invention adopts a dynamic routing mechanism based on feature importance scoring to automatically determine the transmission path of each feature in different layers of the encoder, thereby enhancing the model's ability to reconstruct the original data and reducing information loss.

[0137] Specifically, the training process of the autoencoder algorithm based on regularization loss is as follows Figure 4 shown.

[0138] S401, initializing the weights and biases of the autoencoder. In one embodiment, the initialization method is expressed as:

[0139]

[0140] In the formula, The first The initial weights of the layer; The first The initial bias of the layer; The first The number of neurons in the layer; The first The number of neurons in the layer; The generated shape is Standard normal distribution random numbers; The generated length is The zero vector of .

[0141] S402, the input data is passed through each layer of the encoder, and each layer makes dynamic routing decisions based on the output of the previous layer and the feature importance score of the current layer, selectively passing the most informative features, expressed as:

[0142]

[0143]

[0144] In the formula, The first The linear transformation output of the layer; is the output feature of the activation function of the autoencoder; The first The weight of the layer; The first Bias of the layer; is the ReLU activation function; The first The input features of the layer; represents the dynamic routing function based on feature importance; Calculate the importance score function of the output feature of the activation function of the autoencoder.

[0145] In one embodiment, the dynamic routing function integrates the activation outputs of each feature by weighted average, where the weight is determined by the importance score of the feature, and the calculation method is expressed as:

[0146] Furthermore, the calculation method of the importance score of the output feature of the activation function of the autoencoder is expressed as:

[0147] In the formula, It is the autoencoder Tier The activation output of a neuron, It is the autoencoder Tier The activation output of each neuron; It is the autoencoder Tier The weight of the importance of the activation output feature of the neuron is obtained through training and represents the The importance of a feature; It is the autoencoder Tier The weight of the importance of the activation output feature of the neuron is obtained through training and represents the The importance of a feature; It is The total number of neurons in the layer.

[0148] S403, in the decoder stage, the features from different coding layers are used to reconstruct the data and calculate the reconstruction error. In the training process, the calculation method of the loss function of the autoencoder is expressed as:

[0149] In the formula, is the loss function of the autoencoder; The number of samples entered for the current batch; is the first input to the autoencoder samples; After reconstruction samples; is the number of layers of the autoencoder, is the L2 norm, A regularization loss term that represents the autoencoder weights; is the regularization loss weight of the autoencoder. Preferably, Set to 0.3.

[0150] S404, calculate the gradient of the loss function to each parameter of the autoencoder through the back propagation algorithm, and update the weight and bias of the autoencoder. The update method is expressed as:

[0151]

[0152]

[0153]

[0154] In the formula, The first The update increment of the layer weights; The first The update increment of the layer's bias; is the learning rate of the autoencoder; is the updated autoencoder The weight of the layer; is the updated autoencoder Bias of the layer; The first Sparsity measure of layer neurons; is the pruning threshold of the autoencoder; is an indicator function, which takes 1 when the condition in the brackets is met, otherwise it takes 0. Preferably, It is set to 0.8, indicating that weights will be pruned when the sparsity measure is above 80%.

[0155] In one embodiment, the computational efficiency and generalization ability of the model are improved by dynamically pruning redundant weights, that is, the weights corresponding to neurons with smaller activation values ​​in most cases will be dynamically pruned, thereby simplifying the model structure. Specifically, the importance of neurons is judged by statistically analyzing the sparsity of activation values. The calculation method of the sparsity measure of layer neurons is expressed as:

[0156] In the formula, For the autoencoder Sparsity measure of layer neurons; For the The sample in The activation value of the layer; is the sparsity threshold of the autoencoder. Preferably, Set to 0.1.

[0157] S405, repeat the above steps until the preset stop iteration condition is met, which means the model training is completed. In one embodiment, the preset stop iteration condition is reaching a preset maximum number of iterations, preferably, the preset maximum number of iterations is set to 1000 times.

[0158] S5. Classifier model training The reduced-dimensional data is input into the classifier to train the classifier model. The present invention adopts a high-order neural network algorithm as the classifier model. On the basis of the traditional high-order neural network, in order to overcome the gradient vanishing problem in training, the present invention dynamically adjusts the flow direction and size of the gradient during each back propagation to ensure that the gradient can be effectively transmitted to all network layers, thereby improving the classification accuracy.

[0159] Specifically, the training process of the high-order neural network algorithm is as follows: S501, initializing the parameters of the high-order neural network. In one embodiment, the initialization method is expressed as:

[0160]

[0161] In the formula, is the initial weight of the high-order neural network at the beginning of training; is the initial bias of the high-order neural network at the beginning of training; It means the mean is 0 and the variance is Preferably, Set to 0.01.

[0162] S502, the features after input dimension reduction are subjected to non-uniform feature mapping and forward propagated through the network layer by layer. The output of each layer is used as the input of the next layer, which is expressed as:

[0163]

[0164] In the formula, For high-order neural networks The output of the layer is used as the output of the high-order neural network. The input of the layer; For high-order neural networks The weight of the layer, For high-order neural networks Bias of the layer; is the Sigmoid activation function; It is a high-order neural network The weighted input of the layer, For high-order neural networks The output of the layer.

[0165] S503. In the output layer of the high-order neural network, the Softmax function is used to calculate the predicted probability of each category, and the error between the true label and the predicted probability is calculated by combining the adaptive regularization term with the cross entropy loss function. The calculation method of the loss function is expressed as:

[0166] In the formula, It is One-hot encoding of the true labels; The model predicts The probability of each category; is the total number of categories; is the total number of layers of the high-order neural network; For high-order neural networks Adaptive regularization term for iterations; is the loss function of the high-order neural network; is the number of iterations up to the current time.

[0167] The adaptive regularization term dynamically adjusts the loss function according to the state of the network at different training stages. This mechanism can reduce the risk of overfitting in the early training stage and maintain the robustness of the model in the later training. Specifically, the calculation method of the adaptive regularization term is expressed as:

[0168] In the formula, is the initial adaptive regularization term coefficient, is the parameter for the adaptive regularization term, For the The squared norm of the layer weight updates, is the total number of layers of the high-order neural network, To avoid a small constant for division by zero. Preferably, Set to 0.3, Set to 0.001.

[0169] Furthermore, the model predicts The calculation method of the probability of each category is expressed as:

[0170] In the formula, is the output of the last layer of the network.

[0171] S504, calculate the gradient of the loss function for each parameter, and update the weight and bias parameters of the high-order neural network. The update method is expressed as:

[0172]

[0173] In the formula, is the parameter update operation, is the update increment of the high-order neural network weight parameters, is the update increment of the bias parameters of the high-order neural network.

[0174] In one embodiment, the present invention uses a gradient correction strategy to calculate the parameter increment of a high-order neural network, which is expressed as:

[0175] In the formula, is the learning rate of the high-order neural network, is the gradient of the loss function of the high-order neural network with respect to the weight parameters, is the element-wise multiplication operation, is a symbolic function; is the maximum value function; Accumulate impact factors for gradients to smooth the update process; is the weight of the previous layer of high-order neural network. Preferably, Set to 0.01, Set to 0.5.

[0176] Similarly, the update of bias parameters of high-order neural networks is also calculated using the gradient correction strategy.

[0177] S505, repeat the above steps until the preset stop iteration condition is met, which means that the model training is completed. In one embodiment, the preset stop iteration condition is reaching a preset maximum number of iterations, preferably, the preset maximum number of iterations is set to 1000 times.

[0178] S6, Locomotive driver fatigue detection and recognition The trained model is used to process new samples, monitor the fatigue status of the locomotive driver in real time, and take corresponding safety measures accordingly. In one embodiment, the collected raw data is input into the trained feature extraction and feature dimension reduction model for feature processing, and further, the processed features are input into the classifier model for classifier training to obtain the classification results. In this embodiment, the classification categories include: Level 0: The driver is in good condition and shows no signs of fatigue; Level 1: The driver is slightly fatigued; Level 2: The driver is moderately fatigued; Level 3: The driver is severely fatigued.

[0179] Furthermore, once fatigue is detected, the system will automatically trigger a safety warning, such as an audible alarm or seat vibration, to wake the driver or notify the control center to take further action.

[0180] A second aspect of the embodiments of the present invention provides a device for detecting a fatigue state of a locomotive driver. Figure 5FIG. 1 is a schematic diagram showing a device for detecting a fatigue state of a locomotive driver provided by an embodiment of the present invention. Figure 5 As shown, it includes: a first module 101, establishing a user image sample data set of the fatigue state of a locomotive driver and marking the fatigue level; a second module 102, inputting the user image sample data set into a generative adversarial network model, training the generator and discriminator of the generative adversarial network model for data expansion; a third module 103, inputting the user image sample data set after data expansion into a feature extraction model, a feature dimensionality reduction model and a classifier model for model training, and obtaining a trained user image sample model; a fourth module 104, based on the trained user image sample model, real-time monitoring of the locomotive driver's fatigue level and providing corresponding safety measures.

[0181] According to another aspect of the present invention, an electronic device is provided. The electronic device may be a server. Figure 6 As shown. The electronic device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the electronic device is used to store data. The network interface of the electronic device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the above-mentioned method for detecting the fatigue state of a locomotive driver is implemented.

[0182] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer-readable storage medium, and its internal structure diagram is shown in FIG. Figure 7 The computer readable storage medium stores a computer program that, when executed by a processor, performs the above method for detecting fatigue status of a locomotive driver.

[0183] Finally, it should be noted that a person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program of the method for setting system parameters can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium of the program can be a disk, an optical disk, a read-only storage memory (ROM) or a random access memory (RAM), etc. The above-mentioned computer program embodiment can achieve the same or similar effect as any of the above-mentioned method embodiments corresponding thereto.

[0184] In addition, the method disclosed in the embodiment of the present invention can also be implemented as a computer program executed by a processor, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the above functions defined in the method disclosed in the embodiment of the present invention are performed.

[0185] In addition, the above method steps and system units may also be implemented using a controller and a computer-readable storage medium for storing a computer program that enables the controller to implement the above steps or unit functions.

[0186] It will also be appreciated by those skilled in the art that various exemplary logic blocks, modules, circuits and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, a general description has been given to the functions of various schematic components, blocks, modules, circuits and steps. Whether this function is implemented as software or hardware depends on specific applications and the design constraints imposed on the entire system. Those skilled in the art can implement the function in various ways for each specific application, but this implementation decision should not be interpreted as causing a departure from the disclosed scope of the embodiments of the present invention.

[0187] In one or more exemplary designs, the functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted through a computer-readable medium as one or more instructions or codes. Computer-readable media include computer storage media and communication media, which include any media that facilitate the transfer of computer programs from one location to another. The storage medium may be any available medium that can be accessed by a general or special-purpose computer. As an example and not by way of limitation, the computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to carry or store the required program code in the form of instructions or data structures and can be accessed by a general or special-purpose computer or a general or special-purpose processor. In addition, any connection may be appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves are used to send software from a website, server, or other remote source, the above-mentioned coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are all included in the definition of the medium. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0188] The above are exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications may be made without departing from the scope disclosed in the embodiments of the present invention as defined in the claims. The functions, steps and / or actions of the method claims according to the disclosed embodiments described herein do not need to be performed in any particular order. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in individual form, they may also be understood as multiple unless explicitly limited to the singular.

[0189] It should be understood that, as used herein, the singular forms "a", "an" are intended to include the plural forms as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations including one or more of the associated listed items.

[0190] The serial numbers of the embodiments disclosed in the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0191] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0192] A person skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the disclosure of the embodiments of the present invention (including the claims) is limited to these examples; under the concept of the embodiments of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes in different aspects of the above embodiments of the present invention, which are not provided in detail for the sake of simplicity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the embodiments of the present invention.

Claims

1. A method for detecting a locomotive driver's fatigue state, characterized in that: The following steps are involved: S1. Establish a user image sample dataset of locomotive driver fatigue status and mark the fatigue level; S2, inputting the user image sample data set into a generative adversarial network model, and training a generator and a discriminator of the generative adversarial network model to perform data expansion; S3, input the user image sample data set after data expansion into the feature extraction model, feature dimensionality reduction model and classifier model respectively, perform feature extraction model training based on a fully connected neural network, perform feature dimensionality reduction model training based on an autoencoder algorithm with regularization loss, and perform classifier model training based on a high-order neural network algorithm, to obtain a trained user image sample model; S4. Based on the trained user image sample model, the fatigue level of the locomotive driver is monitored in real time, and corresponding safety measures are given; Wherein, training the generator and the discriminator of the generative adversarial network model includes adjusting the parameters of the generator and the discriminator using the gradient descent method, and updating the parameters of the generator and the discriminator in combination with a dynamic adjustment function; Dynamically adjust the training focus of the generator according to the quality of the user image sample data set and the discrimination result of the discriminator, and adopt a dynamic optimization strategy to dynamically adjust the training focus of the generator according to the feedback of the discriminator; After each training cycle, the generated images are evaluated at the feature level, and the loss function is adjusted according to the feedback from the fine-grained feature module, giving priority to improving areas where features are insufficiently expressed.

2. The method for detecting the fatigue state of a locomotive driver according to claim 1, characterized in that: The parameters of the generator and discriminator are expressed as: In the formula, is the loss function of the generator, is the loss function of the discriminator, is the learning rate of adversarial training, For the The parameters of the generator for the iterations, For the The parameters of the discriminator for the iteration, For the The parameters of the generator for the iterations, For the The parameters of the discriminator for the iteration, is the gradient of the generator’s loss function with respect to the generator’s parameters, is the gradient of the discriminator's loss function with respect to the discriminator parameters, is a dynamic adjustment function, is the gradient of the composite loss function with respect to the generator parameters, is the composite loss function, is the gradient of the composite loss function with respect to the discriminator parameters.

3. The method for detecting the fatigue state of a locomotive driver according to claim 1, characterized in that: The calculation method of the dynamic adjustment function is expressed as: In the formula, represents the generated image data, is the feature recalibration factor, which represents the adjustment intensity of dynamic adjustment; is the symbol of partial derivative; The dynamic adjustment function uses the feature recalibration factor to influence the discrimination focus of the discriminator, so as to prompt the generator to focus on features that are highly related to the fatigue state when generating new images; The calculation method of the conditional probability distribution of generated image and real image features is expressed as: In the formula, represents the real image data, yes and The probability of the intersection of the two is is the marginal probability of the real image feature, Represents the conditional probability distribution of generated image and real image features; The feature recalibration factor is calculated based on the conditional probability distribution of the generated image and the real image features. The calculation method is expressed as: In the formula, is a scale parameter used to adjust the effect of recalibration.

4. The method for detecting the fatigue state of a locomotive driver according to claim 1, characterized in that: The calculation method of the feature enhancement function is expressed as: In the formula, is the feature enhancement function, represents the real image data, is the first Features, is the real image Features, It is The feature weight of each feature, is the similarity calculation function, is the feature enhancement coefficient; is the number of features of the image, that is, the number of pixels; The calculation method of the similarity calculation function is expressed as: In the formula, Represents the difference between the generated image and the real image in The square of the Euclidean distance of feature dimensions, It is the L2 norm, and its calculation method is the same as that of the Euclidean distance; It is The scale parameter of each feature is a training parameter, which is adjusted by the gradient descent method to adjust the contribution of different features to the overall similarity calculation; Represents an exponential function with a natural constant as its base.

5. The method for detecting the fatigue state of a locomotive driver according to claim 4, characterized in that: The feature enhancement function is characterized as a pixel-level loss, and the composite loss function is adjusted by combining the pixel-level loss and the feature-level loss. The calculation method of the composite loss function is expressed as: In the formula, represents the feature-level loss, is the composite loss balance coefficient; The calculation method of feature level loss is expressed as: In the formula, and are the generated and real images, respectively. The sample Features The number of samples entered for the current batch; is a normalization constant to ensure comparability of losses between different features.

6. The method for detecting the fatigue state of a locomotive driver according to claim 1, characterized in that: The feature extraction model training based on the fully connected neural network in step S3 is specifically as follows: S301, initialize the parameters of the neural network, and set the initial adjustment frequency and adjustment amplitude, expressed as: In the formula, To adjust the frequency, To adjust the initial value of the frequency, To adjust the amplitude, To adjust the initial value of the amplitude, It is a hyperparameter that is adaptively adjusted according to the network depth. is a hyperparameter that is adaptively adjusted according to the network width. To adjust the weight factor of the amplitude, is the hyperbolic tangent function, Assignment parameters to adjust the amplitude; S302, input samples and perform forward propagation, calculate the error between the network output and the actual output, expressed as: In the formula, is the Sigmoid activation function; is the output of the neural network, which is obtained by calculating the output of the last layer of neurons in the neural network through Softmax; is the weight of the neural network, is the bias of the neural network, is the error of the neural network, is the true label of the sample; S303, dynamically calculate the resonance factor based on the error size for subsequent frequency and amplitude adjustment, and the calculation method is expressed as: In the formula, is the resonance factor, is a hyperparameter of the sensitivity of the reaction error to the resonance adjustment; S304, update the adjustment frequency according to the resonance factor adjustment parameter, so that it increases when the error is large and decreases when the error is small. The calculation method is expressed as: In the formula, For the The adjustment frequency of the iterations, For the The adjustment frequency of the iterations, is the frequency adjustment rate; S305, update the size of the adjustment amplitude to ensure that the update step size matches the error and avoid oscillation caused by too large a step size. The calculation method of the adjustment amplitude is expressed as: In the formula, For the The adjustment amplitude of the iteration, For the The adjustment amplitude of the iteration, is the adjustment angle of the amplitude; S306. Calculate and adjust the phase using the phase correction mechanism to ensure synchronization of parameter updates and optimize the convergence path. The adjustment method is expressed as: In the formula, For the The adjustment phase of the iteration, For the The adjustment phase of the iteration, is the phase update rate, Indicates the current iteration number; S307, updating the weights and biases of the neural network according to the adjustment frequency, the adjustment amplitude and the adjustment phase, the update method is expressed as: In the formula, For the The weights of the neural network at the iteration, For the The bias of the neural network at the iteration, For the The weights of the neural network at the iteration, For the The bias of the neural network at the iteration, is the gradient of the neural network error with respect to the weight parameters, is the gradient of the neural network error with respect to the bias parameter; S308, repeat the above steps until the preset stop iteration condition is met, which means that the model training is completed.

7. The method for detecting the fatigue state of a locomotive driver according to claim 3, characterized in that: The calculation method of the importance score of the output feature of the activation function of the autoencoder is expressed as: In the decoder stage, the features from different encoding layers are used to reconstruct the data and calculate the reconstruction error. During the training process, the loss function of the autoencoder is calculated as follows: In the formula, is the loss function of the autoencoder; The number of samples entered for the current batch; is the first input to the autoencoder samples; After reconstruction samples; is the number of layers of the autoencoder, is the L2 norm, A regularization loss term that represents the autoencoder weights; is the regularization loss weight of the autoencoder.

8. The method for detecting the fatigue state of a locomotive driver according to claim 1, characterized in that: The importance of neurons is determined by counting the sparsity of the activation values. The calculation method of the sparsity measure of layer neurons is expressed as: In the formula, For the autoencoder Sparsity measure of layer neurons; For the The sample in The activation value of the layer; is the sparsity threshold of the autoencoder.

9. The method for detecting the fatigue state of a locomotive driver according to claim 1, characterized in that: The classifier model training based on the high-order neural network algorithm includes: In the output layer of the high-order neural network, the Softmax function is used to calculate the predicted probability of each category, and the error between the true label and the predicted probability is calculated by combining the adaptive regularization term with the cross entropy loss function. The calculation method of the loss function is expressed as: In the formula, It is One-hot encoding of the true labels; The model predicts The probability of each category; is the total number of categories; is the total number of layers of the high-order neural network; For high-order neural networks Adaptive regularization term for iterations; is the loss function of the high-order neural network; is the number of iterations up to the current time; The adaptive regularization term dynamically adjusts the loss function according to the state of the network at different training stages. This mechanism can reduce the risk of overfitting in the early training stage and maintain the robustness of the model in the later training. Specifically, the calculation method of the adaptive regularization term is expressed as: In the formula, is the initial adaptive regularization term coefficient, is the parameter for the adaptive regularization term, For the The squared norm of the layer weight updates, is the total number of layers of the high-order neural network, A small constant to avoid division by zero. Furthermore, the model predicts The calculation method of the probability of each category is expressed as: In the formula, is the output of the last layer of the network.

10. The method for detecting the fatigue state of a locomotive driver according to claim 1, characterized in that: The gradient correction strategy is used to calculate the parameter increment of the high-order neural network, which is expressed as: In the formula, is the learning rate of the high-order neural network, is the gradient of the loss function of the high-order neural network with respect to the weight parameters, is the element-wise multiplication operation, is a symbolic function; is the maximum value function; Accumulate impact factors for gradients to smooth the update process; is the weight of the previous layer of high-order neural network.

Images