Face anonymization fatigue detection method and system based on generative model

Abstract

This invention relates to the fields of intelligent transportation systems and artificial intelligence, aiming to provide a generative model-based method and system for facial anonymization fatigue detection. The invention anonymizes driver facial images from videos using a generative model, removing features related to identity information while retaining key facial features for fatigue detection. Simultaneously, it designs downstream fatigue detection algorithms and jointly optimizes both algorithms to form an end-to-end system, ensuring that the anonymized image does not affect detection accuracy. This invention can replace the driver's real identity information while retaining the key facial features required for fatigue detection, generating a fake image without identity features. This effectively protects the driver's privacy while ensuring the usability of the detection data. At the same time, it improves the robustness and accuracy of the detection system, significantly reduces false alarms, and makes it more adaptable and stable in complex driving environments.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation systems and artificial intelligence technology, specifically relating to a fatigue detection method and system based on generative models and facial anonymization; aiming to perform fatigue detection without disclosing driver identity information based on video anonymization and face replacement technology using generative models. Background Technology

[0002] Road traffic safety has always been a key issue in global public safety. Despite extensive efforts by various countries to improve road safety, the number of fatalities and injuries caused by traffic accidents remains high, resulting in significant economic losses. Multiple studies have shown that driver fatigue is a major cause of traffic accidents; therefore, accurately and effectively detecting driver fatigue is crucial for improving traffic safety.

[0003] Existing driver fatigue detection methods can be broadly categorized into two types: vision-based and non-visual methods. Non-visual methods typically rely on the driver's physiological signals, such as electroencephalography (EEG), electrocardiography (ECG), electromyography (EMG), and electrooculography (EOG), or use vehicle status data to analyze the driver's fatigue state. However, monitoring physiological signals has limitations. It requires wearable sensors to collect data, which is invasive, increases driver discomfort, and the equipment is expensive, hindering widespread adoption. Furthermore, while methods using vehicle data for fatigue detection offer readily available data, their performance is poor and they cannot accurately assess the driver's condition.

[0004] Vision-based methods, thanks to the rapid development of computer vision technology, have become the mainstream approach for fatigue detection. These methods capture the driver's facial features through cameras and determine the driver's fatigue level based on features such as eye closure and facial expressions. This method is non-invasive, relatively simple to deploy, and offers high detection accuracy. However, existing vision detection methods involve real-time image acquisition of the driver, posing significant privacy concerns. Even without physical contact, the driver's subjective awareness of being monitored by a camera may cause psychological discomfort, and there is also a risk of image privacy breaches.

[0005] Currently, research combining vision-based driver fatigue detection and image / video anonymization has not received enough attention, and even publicly available technological achievements have their own technical shortcomings.

[0006] For example, patent application CN115908625A proposes a cyclic reversible anonymized face synthesis method to prevent identity leakage. This method utilizes generative adversarial networks (GANs) to anonymize face images while ensuring the effectiveness of the anonymized faces. However, with large datasets, the efficiency of the clustering operations used in this method becomes a bottleneck, making it difficult to capture the complex nonlinear features in face images. The acquired features may lead to blurred or overly averaged images, limiting the realism and detail of the images. Furthermore, the system's reversibility, where image information can still be recovered after anonymization, may compromise the user's perception of privacy.

[0007] Some researchers have attempted to protect driver privacy to some extent by extracting only task-relevant facial features. For example, "de-identification" fatigue detection methods extract and use only the facial area around the eyes, masking everything else in the background. However, this method remains difficult to generalize in practice. Preserving some foreground features may not completely remove identity information, while excessive background blurring can affect the model's understanding of context, leading to decreased detection accuracy. Furthermore, its adaptability to different lighting conditions and occlusion scenarios is limited, significantly reducing data availability without achieving good detection accuracy.

[0008] Therefore, it is necessary to propose new solutions that better meet the needs of computer vision tasks in specific domains, ensuring that anonymized images can be used efficiently in these tasks without compromising their performance. Summary of the Invention

[0009] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a face anonymization fatigue detection method and system based on generative models.

[0010] To solve the above-mentioned technical problems, the solution adopted by the present invention is:

[0011] A face anonymization fatigue detection method based on a generative model is provided, the method comprising:

[0012] The system acquires dynamic video of the driver's face and segments it according to a preset duration. Each frame of the video slice within a preset time interval is used as the original image for subsequent processing.

[0013] Key facial features and pose geometry heatmaps are extracted simultaneously from the acquired raw images;

[0014] A latent encoding is generated using a generator from a Generative Adversarial Network (GAN), and a new facial image is further generated based on this latent encoding and the compressed information contained in the facial key features and pose geometry heatmap. Anonymization is achieved while preserving key features by blurring or replacing features related to personal identity and retaining core information related to fatigue detection.

[0015] The anonymized image data is preprocessed to obtain the global and local feature vectors of each frame. After appropriate splicing, the overall feature representation of each frame is formed. Then, the overall feature representation is arranged and combined with the feature information in the spatial and temporal dimensions to generate a spatiotemporal feature matrix.

[0016] Spatiotemporal attention network is used to extract temporal information and spatial features from spatiotemporal feature matrix, adaptively capture the dynamic correlation between various time points and key features of different spatial locations, and output a one-dimensional spatiotemporal joint feature vector for fatigue judgment.

[0017] The spatiotemporal joint feature vector is subjected to feature mapping and probability calculation. Based on the set threshold, the probability value is classified and the prediction result of fatigue state is output. If it is determined that the driver is in a fatigue state, a fatigue alarm is issued. If it is determined that the driver is not in a fatigue state, all the above operations are repeated.

[0018] As a preferred embodiment of the present invention, a facial key feature extraction module is constructed, and the key point detection and feature extraction model contained in the module is pre-trained and then used to extract facial key points and related features from the original image; the key points and related features include the position and size of the driver's eyes, pupils, and mouth, as well as the degree of closure of the eyes and mouth; a pose geometry heatmap extraction module is constructed, and the pose estimation and key point detection deep learning network contained in the module is pre-trained and then used to extract the pose geometry heatmap; the features of the pose geometry heatmap include the driver's head pose, facial orientation, and facial geometry information related to fatigue detection.

[0019] As a preferred embodiment of the present invention, when generating a new facial image, a generator is used to adjust some dimensions in the latent encoding, blurring or replacing features related to personal identity; then, based on the compressed information contained in the input facial key features and pose geometry heatmap, the core information related to fatigue detection is retained in a controlled manner; the generator has learned the rules of feature transformation in advance through pre-training.

[0020] As a preferred embodiment of the present invention, the preprocessing of anonymized image data to obtain global feature vectors specifically includes: firstly, using a pre-trained keypoint detection model to detect facial keypoints; then, performing temporal alignment and normalization on all keypoints; next, constructing a feature matrix based on the facial keypoint data of each frame, representing the keypoint information as a two-dimensional matrix; and then using a dimensionality reduction algorithm to perform spatial filtering on the feature matrix, enhancing and reducing the dimensionality of the input facial keypoint features, and finally obtaining the global feature vector of each frame of the anonymized image.

[0021] As a preferred embodiment of the present invention, when preprocessing anonymous image data to obtain local feature vectors, the specific steps include: extracting the bounding boxes of key regions from each frame of the image, extracting the corresponding local facial regions according to the size and position of the bounding boxes, and dividing them into multiple small blocks; applying a feature embedding algorithm to the local facial regions, and mapping each small block into a feature vector through linear mapping; subsequently, concatenating the feature vectors of the local facial regions in each frame to form the local feature vector of that frame of the image.

[0022] As a preferred embodiment of the present invention, a pre-trained spatiotemporal attention network is used to extract spatiotemporal features of video images from the spatiotemporal feature matrix. The temporal information and spatial features of each frame are globally modeled through the spatiotemporal attention mechanism. Based on the attention mechanism, the dynamic correlation between various time points and key features of different spatial locations are adaptively captured. The spatial features include head posture and the degree of closure of the eyes and mouth. The temporal information includes head movement, head posture changes, blinking frequency, yawning and gaze shift information.

[0023] As a preferred embodiment of the present invention, the spatiotemporal joint feature vector is processed by the classification module, including: mapping the input spatiotemporal joint features to a new feature space in several fully connected layers of the classification module, and generating binary classification probability values ​​in the output layer of the classification module through the Sigmoid activation function.

[0024] This invention further provides a face anonymization fatigue detection system based on a generative model, comprising: a face anonymization model and a fatigue detection model, the former including a facial key feature extraction module, a pose geometry heatmap extraction module, and an anonymized frame generation module, and the latter including a data processing module, a feature extraction module, and a fatigue classification module; wherein,

[0025] The facial key feature extraction module includes a key point detection and feature extraction model, which is used to extract facial key features from each frame of the original image of a video slice.

[0026] The pose geometry heatmap extraction module includes a deep learning network for pose estimation and keypoint detection, used to extract pose geometry heatmaps from each frame of the original image of a video slice.

[0027] The anonymized frame generation module has a built-in pre-trained generative adversarial network to generate a latent code, and further generates a new facial image based on the latent code and the compressed information contained in the facial key features and pose geometry heatmap; by blurring or replacing features related to personal identity and retaining the core information related to fatigue detection, anonymization is achieved while retaining key features.

[0028] The data processing module is used to preprocess the anonymized image data to obtain the global feature vector and local feature vector of each frame of the image. After appropriate splicing, the overall feature representation of each frame of the image is formed. Then, the overall feature representation is arranged and combined with the feature information in the spatial and temporal dimensions to generate a spatiotemporal feature matrix.

[0029] The feature extraction module uses a spatiotemporal attention network to extract temporal information and spatial features from the spatiotemporal feature matrix, adaptively capturing the dynamic correlation between various time points and key features of different spatial locations, and outputting a one-dimensional spatiotemporal joint feature vector for fatigue judgment.

[0030] The fatigue classification module is used to perform feature mapping and probability calculation on the spatiotemporal joint feature vector, make classification decisions on the probability values ​​according to the set threshold, and output the predicted results of fatigue state.

[0031] The present invention also provides a computer device, the computer device including a memory and a processor connected in a communicative manner; wherein, the memory is used to store program data, and the processor is used to execute the program data to implement the aforementioned face anonymization fatigue detection method based on generative model;

[0032] The present invention further provides a computer-readable storage medium, characterized in that the computer-readable storage medium includes stored program data; when the program data is run by a processor, the computer device where the storage medium is located is controlled to execute the aforementioned face anonymization fatigue detection method based on generative model.

[0033] Compared with the prior art, the technical advantages of the present invention are:

[0034] 1. This invention, by applying facial anonymization technology, can replace the driver's real identity information while retaining the key facial features required for fatigue detection, generating a false image without identity features, thereby effectively protecting the driver's privacy while ensuring the availability of detection data.

[0035] 2. This invention focuses on utilizing the key information retained after anonymization processing to improve the robustness and accuracy of the detection system, significantly reduce false alarms, and enable it to have better adaptability and stability in complex driving environments.

[0036] 3. Based on the organic combination of anonymization and fatigue detection, this invention constructs a highly cohesive and loosely coupled end-to-end system; it achieves a balance between privacy protection and fatigue detection performance, effectively improving the overall performance and application value of the system.

[0037] 4. This invention successfully solves the problems of driver privacy leakage and low detection accuracy in traditional fatigue detection methods. Compared with existing methods based on direct video capture and analysis, this invention can effectively remove identity-related information, retain only necessary detection features, and reduce the risk of privacy leakage.

[0038] 5. This invention combines a spatiotemporal attention network based on deep learning, which significantly reduces the false alarm rate while improving fatigue detection accuracy, and exhibits better robustness, especially in complex driving scenarios. By assessing fatigue status in real time, this invention can promptly alert the driver to avoid traffic accidents, thereby improving driving safety. Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating the implementation of the fatigue detection method described in this invention.

[0040] Figure 2 This is a schematic diagram of the fatigue detection system described in this invention. Detailed Implementation

[0041] First, it should be noted that this invention relates to artificial intelligence technology, specifically an application of computer technology in the field of road traffic safety. The implementation of this invention involves the application of multiple software functional modules. The applicant believes that, after carefully reading the application documents and accurately understanding the implementation principles and objectives of this invention, and in conjunction with existing publicly known technologies, those skilled in the art can fully utilize their software programming skills to implement this invention. The aforementioned software functional modules include, but are not limited to: a face anonymization model, a fatigue detection model, a facial key feature extraction module, a pose geometry heatmap extraction module, an anonymous frame generation module, a data processing module, a feature extraction module, a fatigue classification module, generative adversarial networks, deep learning networks, key point detection and feature extraction models, pose estimation and key point detection models, etc. All those mentioned in this application fall within this scope, and the applicant will not list them all further.

[0042] Those skilled in the art will understand that, besides implementing a portion of the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, enabling the system and its various devices, modules, and units to function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered both software modules implementing the method and structures within the hardware component.

[0043] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0044] To address the shortcomings of existing technologies, this invention proposes a driver fatigue detection method based on facial anonymization. The core idea is to anonymize driver facial images from videos using a generative model, removing features related to identity information while retaining key facial features for fatigue detection. Simultaneously, downstream fatigue detection algorithms are designed and optimized in a targeted manner, forming an end-to-end integrated system that ensures the anonymized image does not affect detection accuracy. This method effectively reduces the risk of privacy leakage and improves the accuracy and robustness of the detection system.

[0045] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0046] I. The present invention proposes a driver fatigue detection method based on face anonymization, the specific implementation steps of which are as follows:

[0047] (1) Raw data acquisition: The driver's dynamic video is acquired by an RGB camera installed below the rearview mirror, and the video is segmented into multiple video slices according to a preset duration t. Each frame f in the video slice within the preset time interval t is used as the input raw image for subsequent processing.

[0048] (2) Facial Key Feature Extraction: Each original frame f in the video slice is input into the face anonymization model, which has a built-in facial key feature extraction module, which includes a key point detection and feature extraction model. Key points and related features are extracted using the pre-trained key point detection and feature extraction model, including the position and size of the driver's eyes, pupils, and mouth, as well as the degree of eye and mouth closure, etc.

[0049] (3) Pose geometry heatmap extraction: Each original frame f in the video slice is input into the face anonymization model, which has a built-in pose geometry heatmap extraction module, including pose estimation and key point detection deep learning networks. The driver pose geometry heatmap m is extracted using the pre-trained pose estimation and key point detection deep learning network. This heatmap contains features related to fatigue detection, such as driver head pose, facial orientation, and facial geometric information.

[0050] Note that step (3) and step (2) are executed in parallel.

[0051] (4) Face anonymization processing:

[0052] Image and video anonymization techniques can be broadly categorized into two types: those based on Generative Adversarial Networks (GANs) and those based on diffusion models. Based on the applicant's research, GAN models generally outperform diffusion models in video anonymization and driver fatigue detection tasks. Therefore, this application employs Generative Adversarial Networks to anonymize driver videos. The "based on generative model" mentioned in the invention title refers to a "face anonymization model based on a built-in Generative Adversarial Network."

[0053] As a specific application example, the model for generating anonymized faces can use a pre-trained StyleGAN3. The key facial features and pose geometry heatmaps extracted in the previous steps are concatenated with the latent encoding generated by the model. This concatenation is then input into the StyleGAN3 generator to generate a new anonymized image, ensuring that the key facial features and head pose information after anonymization are stable and consistent with the original image. The generator of this model generates a new replacement face image by combining the compressed information from the latent encoding, key facial features, and pose geometry heatmaps, anonymizing the driver's face while preserving key features.

[0054] Specifically, a generator using a pre-trained Generative Adversarial Network (GAN) generates a latent code *c* in the latent space. Then, based on this code, the input facial key features *ε*, and the compressed information contained in the pose geometry heatmap *m*, a new facial image *g(c,ε,m)* is generated. During this process, the generator adjusts some dimensions of the latent code, blurring or replacing features related to personal identity. Simultaneously, it retains core information relevant to fatigue detection, such as eye and mouth features and head pose, in a controlled manner based on the input *ε* and *m*. This generation method ensures that privacy information in the image is hidden or replaced without affecting the subsequent fatigue detection task. In the inference phase, the generator has already learned the rules of feature transformation through pre-training and directly generates anonymized images based on the input features without further adjusting model parameters. Through GAN processing, the generated anonymized image is obtained, ensuring that privacy information of facial features is hidden while retaining the key information required for fatigue detection.

[0055] (5) Repeat steps (1)-(4) until all frames in the given video slice are anonymized.

[0056] (6) Enter the fatigue detection stage: Input the anonymized image into the fatigue detection model for further processing.

[0057] (7) Fatigue detection data preprocessing: After the anonymized image is input into the fatigue detection model, the data processing module first preprocesses the video data.

[0058] First, a pre-trained keypoint detection model is used to detect facial keypoints. Then, all keypoints are temporally aligned and normalized to ensure data consistency and usability; frame dropping interpolation is performed when necessary. Next, a feature matrix is ​​constructed based on the facial keypoint data of each frame, representing the keypoint information of each frame as a two-dimensional matrix. To improve the discriminative ability between samples of different fatigue states, a dimensionality reduction algorithm is used to spatially filter the feature matrix, thereby enhancing and reducing the dimensionality of the input facial keypoint features, resulting in a global feature vector for each frame.

[0059] Simultaneously, bounding boxes of key regions such as the eyes and mouth are extracted from each frame of the video. Based on the size and position of these bounding boxes, corresponding local facial regions are extracted. These regions correspond to the key facial features in step (2) of the method, which are the fatigue detection key features that remain consistent even after anonymization. The extracted local key facial regions (such as the eye and mouth regions) are then divided into multiple small blocks. Feature embedding algorithms are applied to these local facial regions, and these small blocks are mapped into feature vectors through linear mapping. Subsequently, the feature vectors of the local facial regions are concatenated within each frame to form the local feature vector of that frame's image.

[0060] Next, the local feature vectors and global feature vectors of each frame are concatenated within each frame to form the overall feature representation of the image in that frame.

[0061] Finally, the overall feature representations of each frame are arranged to combine feature information from both spatial and temporal dimensions, generating a comprehensive spatiotemporal feature matrix. In this way, the extracted features can effectively capture the spatiotemporal variation patterns of fatigue states, providing more discriminative input features for subsequent classifiers.

[0062] (8) Fatigue detection feature extraction: In order to enhance the accuracy and robustness of fatigue detection, this invention takes into account the spatiotemporal information in the video. That is, in addition to the spatial information of each frame, this invention also focuses on the temporal information of the video, specifically the changes of the driver between different video frames, which are manifested as time-related information such as head movement, head posture changes, gaze shifts, and blink frequency.

[0063] Specifically, the preprocessed spatiotemporal feature matrix from step (7) is input into the feature extraction module. The module uses a Spatio-Temporal Attention Network (STAN) to extract spatiotemporal features from the video. Spatial features include head pose, eye and mouth closure, etc., while temporal features include head movement, head pose changes, blinking frequency, yawning frequency, and gaze shifts. STAN extracts spatiotemporal related features directly from the input sequence through a global spatiotemporal attention mechanism. The temporal information and spatial features of each input frame are globally modeled through the spatiotemporal attention mechanism. The network's attention mechanism adaptively captures the dynamic correlation between different time points and key features at different spatial locations, ensuring the capture of fatigue-related behaviors such as yawning, abnormal eye closure, blinking, head movement, and gaze shifts. This module outputs a one-dimensional spatiotemporal joint feature vector for subsequent final fatigue judgment.

[0064] (9) Fatigue State Classification: After obtaining the spatiotemporal joint feature vector, it is input into the fatigue classification module. This classification module consists of several fully connected layers. Each fully connected layer maps the input features to a new feature space, enhancing the discriminative power of the features. In the final output layer, a binary classification probability value is generated through the Sigmoid activation function to determine the driver's fatigue state. Finally, the system makes a classification decision on the output probability value based on a set threshold, completing the prediction of the fatigue state and outputting whether the driver is fatigued.

[0065] (10) Feedback and Early Warning: If the detection model determines that the driver is fatigued, the system will trigger a fatigue alarm and send a reminder message to the driver. The reminder message may include audio and visual alerts, seat vibration, etc., to help the driver stay alert and prevent dangerous driving behavior.

[0066] (11) If it is determined that the driver is not fatigued, then repeat all the above operations. Repeat steps (1)-(10) while the driver is driving until the fatigue detection system is turned off.

[0067] II. To implement the aforementioned face anonymization driver fatigue detection method, this invention provides a face anonymization driver fatigue detection system formed by two models: a face anonymization model and a fatigue detection model. The former includes a facial key feature extraction module, a pose geometry heatmap extraction module, and an anonymized frame generation module, while the latter includes a data processing module, a feature extraction module, and a fatigue classification module.

[0068] The system includes a facial key feature extraction module, comprising a keypoint detection and feature extraction model, for extracting key facial features from each frame of the original video slice; a pose geometry heatmap extraction module, comprising a pose estimation and keypoint detection deep learning network, for extracting pose geometry heatmaps from each frame of the original video slice; an anonymized frame generation module, incorporating a pre-trained generative adversarial network, for generating latent codes, and further generating new facial images based on these latent codes and the compressed information contained in the facial key features and pose geometry heatmaps; anonymization is achieved by blurring or replacing features related to personal identity while retaining core information related to fatigue detection; and a data processing module for processing the anonymized data. The image data is preprocessed to obtain the global and local feature vectors of each frame. These are then concatenated to form the overall feature representation of each frame. The overall feature representation is then arranged and combined with spatial and temporal feature information to generate a spatiotemporal feature matrix. The feature extraction module uses a spatiotemporal attention network to extract temporal information and spatial features from the spatiotemporal feature matrix, adaptively capturing the dynamic relationships between different time points and key features at different spatial locations, outputting a one-dimensional spatiotemporal joint feature vector for fatigue assessment. The fatigue classification module performs feature mapping and probability calculation on the spatiotemporal joint feature vector, classifies the probability values ​​based on a set threshold, and outputs the predicted fatigue state.

[0069] Furthermore, the present invention also provides a corresponding computer device and a computer-readable storage medium, which implement the various steps of the face anonymization fatigue detection method described above by executing the stored computer program.

[0070] In summary, the driver fatigue detection method proposed in this invention, which combines privacy protection with high-accuracy detection, utilizes facial anonymization technology. This allows the method to retain key facial features required for fatigue detection while replacing the driver's real identity information, generating a false image without any identifying characteristics. This effectively protects the driver's privacy while ensuring the usability of the detection data, and reduces psychological intrusion while maintaining detection accuracy. The fatigue detection algorithm proposed in this invention focuses on utilizing the key information retained after anonymization, improving the robustness and accuracy of the detection system, significantly reducing false alarms, and enabling it to have better adaptability and stability in complex driving environments.

Claims

1. A face anonymization fatigue detection method based on a generative model, characterized in that, The method includes: The system acquires dynamic video of the driver's face and segments it according to a preset duration. Each frame of the video slice within a preset time interval is used as the original image for subsequent processing. Key facial features and pose geometry heatmaps are extracted simultaneously from the acquired raw images; A pre-trained generative adversarial network (GAN) built into the generative model is used to generate a latent code in the latent space. Based on this latent code and the compressed information contained in the facial key features and pose geometry heatmap, features are concatenated and further input into the generator of the GAN to generate a new facial image. During this process, the generator adjusts some dimensions in the latent code, blurring or replacing features related to personal identity. At the same time, based on the input facial key features and pose geometry heatmap, core information related to fatigue detection, including key facial features of the eyes and mouth and head pose, is retained in a controlled manner to ensure that the anonymized key facial features and head pose information are stable and consistent with the original image, thus achieving anonymization while retaining key features. The anonymized image data is preprocessed to obtain the global and local feature vectors of each frame. After appropriate splicing, the overall feature representation of each frame is formed. Then, the overall feature representation is arranged and combined with the feature information in the spatial and temporal dimensions to generate a spatiotemporal feature matrix. Spatiotemporal attention network is used to extract temporal information and spatial features from spatiotemporal feature matrix, adaptively capture the dynamic correlation between various time points and key features of different spatial locations, and output a one-dimensional spatiotemporal joint feature vector for fatigue judgment. The spatiotemporal joint feature vector is subjected to feature mapping and probability calculation. Based on the set threshold, the probability value is classified and the prediction result of fatigue state is output. If it is determined that the driver is in a fatigue state, a fatigue alarm is issued. If it is determined that the driver is not in a fatigue state, all the above operations are repeated.

2. The method according to claim 1, characterized in that, A facial key feature extraction module is constructed, and the key point detection and feature extraction model contained in the module is pre-trained and then used to extract facial key points and related features from the original image; the key points and related features include the position and size of the driver's eyes, pupils, and mouth, as well as the degree of closure of the eyes and mouth; A pose geometry heatmap extraction module is constructed. The pose estimation and key point detection deep learning networks contained in this module are pre-trained and then used to extract pose geometry heatmaps. The features of the pose geometry heatmaps include driver head pose, facial orientation, and facial geometry information related to fatigue detection.

3. The method according to claim 1, characterized in that, When generating a new facial image, the generator adjusts some dimensions in the latent encoding, blurring or replacing features related to personal identity; then, based on the compressed information contained in the input facial key features and pose geometry heatmap, the core information related to fatigue detection is retained in a controlled manner; the generator has been pre-trained to learn the rules of feature transformation.

4. The method according to claim 1, characterized in that, When preprocessing anonymized image data to obtain global feature vectors, the process includes: first, using a pre-trained keypoint detection model to detect facial keypoints; then, performing temporal alignment and normalization on all keypoints; next, constructing a feature matrix based on the facial keypoint data of each frame, representing the keypoint information as a two-dimensional matrix; and finally, using a dimensionality reduction algorithm to perform spatial filtering on the feature matrix, enhancing and reducing the dimensionality of the input facial keypoint features, thereby obtaining the global feature vector for each frame of the anonymized image.

5. The method according to claim 1, characterized in that, When preprocessing anonymous image data to obtain local feature vectors, the process includes: extracting bounding boxes of key regions from each frame of the image; extracting corresponding local facial regions based on the size and position of the bounding boxes; dividing them into multiple small blocks; applying a feature embedding algorithm to the local facial regions and mapping each small block into a feature vector through linear mapping; and then concatenating the feature vectors of the local facial regions within each frame to form the local feature vector of that frame of the image.

6. The method according to claim 1, characterized in that, A pre-trained spatiotemporal attention network is used to extract spatiotemporal features from video images from the spatiotemporal feature matrix. The temporal information and spatial features of each frame are globally modeled through the spatiotemporal attention mechanism. Based on the attention mechanism, the dynamic correlation between various time points and key features of different spatial locations are adaptively captured. The spatial features include head posture and the degree of closure of the eyes and mouth. The temporal information includes head movement, changes in head posture, blink frequency, yawning and gaze shift information.

7. The method according to claim 1, characterized in that, The spatiotemporal joint feature vector is processed using a classification module, including: mapping the input spatiotemporal joint features to a new feature space in several fully connected layers of the classification module, and generating binary classification probability values ​​in the output layer of the classification module through a Sigmoid activation function.

8. A face anonymization fatigue detection system based on a generative model, characterized in that, include: The system includes a face anonymization model and a fatigue detection model. The former includes a facial key feature extraction module, a pose geometry heatmap extraction module, and an anonymized frame generation module, while the latter includes a data processing module, a feature extraction module, and a fatigue classification module. The facial key feature extraction module includes a key point detection and feature extraction model, which is used to extract facial key features from each frame of the original image of a video slice. The pose geometry heatmap extraction module includes a deep learning network for pose estimation and keypoint detection, used to extract pose geometry heatmaps from each frame of the original image of a video slice. The anonymized frame generation module incorporates a pre-trained generative adversarial network (GAN) to generate latent codes in the latent space. Based on these latent codes, compressed information from facial key features and pose geometry heatmaps is concatenated, and the resulting data is further input into the GAN generator to generate new facial images. During this process, the generator adjusts some dimensions in the latent codes, blurring or replacing features related to personal identity. Simultaneously, based on the input facial key features and pose geometry heatmaps, core information related to fatigue detection, including key facial features of the eyes and mouth, as well as head pose, is retained in a controlled manner. This ensures that the anonymized key facial features and head pose information are stable and consistent with the original image, achieving anonymization while preserving key features. The data processing module is used to preprocess the anonymized image data to obtain the global feature vector and local feature vector of each frame of the image. After appropriate splicing, the overall feature representation of each frame of the image is formed. Then, the overall feature representation is arranged and combined with the feature information in the spatial and temporal dimensions to generate a spatiotemporal feature matrix. The feature extraction module uses a spatiotemporal attention network to extract temporal information and spatial features from the spatiotemporal feature matrix, adaptively capturing the dynamic correlation between various time points and key features of different spatial locations, and outputting a one-dimensional spatiotemporal joint feature vector for fatigue judgment. The fatigue classification module is used to perform feature mapping and probability calculation on the spatiotemporal joint feature vector, make classification decisions on the probability values ​​according to the set threshold, and output the predicted results of fatigue state.

9. A computer device, characterized in that, The computer device includes a memory and a processor connected in a communicative manner; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the face anonymization fatigue detection method based on a generative model as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes stored program data; when the program data is run by a processor, the computer device containing the storage medium is controlled to execute the face anonymization fatigue detection method based on any one of claims 1 to 7.

Images