ECLA-HSFPN-fused lightweight fatigue driving detection method

Abstract

The invention relates to the field of image processing, in particular to an ECLA-HSFPN-fused lightweight fatigue driving detection method, which comprises the following steps: collecting a fatigue driving image, labeling the fatigue driving image by using a Labeling tool, and making a related data set; building a network model of fusion of the yolov8 and the ECLA-HSFPN; setting parameters for the improved network model and training the improved network model; constructing a fatigue driving model, and performing model performance evaluation on the test set; deep customization and optimization are carried out on the basis of YOLOv8, in a Head network, ECLA-HSFPN is used for realizing multi-scale feature fusion, high-level features are used as weights to filter low-level feature information, the filtered information is merged with the high-level features, and the feature expression ability of the model is enhanced; under the condition that the detection precision is kept unchanged, the parameter quantity and the operation quantity of the model can be remarkably reduced, and the operation efficiency is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a lightweight fatigue driving detection method integrating ECLA-HSFPN. Background Art

[0002] Traffic accidents have been recognized as the number one public hazard to human life and safety today. With the rapid growth of the number of cars and the continuous advancement of urbanization, people's travel has become more convenient, but it is also accompanied by a series of negative effects, such as frequent traffic accidents, road congestion, environmental pollution, etc. According to data from the Institute of Transportation Medicine of the Ministry of Health, in addition to traffic violations such as drunk driving, speeding, illegal lane changes, and running red lights, serious and major traffic accidents caused by fatigue driving are as high as 43%. Studies have shown that traffic accidents usually occur within a few seconds. If a warning can be given 2 seconds in advance, 92% of traffic accidents can be reduced, and a warning of 0.5 seconds in advance can avoid 73% of traffic accidents.

[0003] Fatigue driving refers to the behavior in which the driver has symptoms such as lack of concentration, slow reaction, unclear thinking, dry eyes, and even drowsiness and loss of consciousness during driving due to factors such as long-term continuous driving, poor sleep quality, and physical fatigue. This is an extremely dangerous driving behavior that can easily cause traffic accidents and seriously endanger the life and property of the driver and others. Therefore, it is of great significance to effectively monitor and warn the driver's fatigue state.

[0004] At present, some existing fatigue detection technologies are mainly divided into two categories: invasive and non-invasive. Among them, the invasive type is based on the driver's physiological signal detection, and the detection methods include: measuring heart rate, pupil reaction, brain waves and other physiological parameters. Secondly, the non-invasive type is based on vehicle behavior detection and driver's facial feature detection. For example, the driver's facial features can be detected by recording and analyzing the driver's operation of the steering wheel, use of brakes and other behaviors, or by using computer vision technology. By collecting the driver's facial image and performing feature analysis on it, such as yawning, blinking, nodding and other features.

[0005] At present, intrusive detection mainly uses physiological sensors to detect changes in the driver's physiological indicators to determine whether the driver has entered a fatigue state. This method is most closely related to fatigue, but the existing contact method not only hinders driving but also has high costs. Among non-invasive detection, vehicle driving characteristics detection can effectively detect the driver's fatigue state, but it requires adding a large number of sensors and other equipment to the vehicle, making the system deployment complex and costly.

[0006] However, there are some problems in using the driver's facial features to detect fatigue driving. Commonly used target detection algorithms include YOLO series, SSD, Faster R-CNN, etc. These algorithms are difficult to meet the requirements of portable hardware for actual vehicle use, and it is difficult to ensure the accuracy and real-time performance of the algorithm. Commonly used lightweight networks include mobilenet, shufflenet, etc. These networks have low complexity and are suitable for running on mobile terminals or other embedded hardware. However, the computational complexity and memory usage of these lightweight networks in fatigue driving detection are large in practical applications, and the number of parameters needs to be improved; moreover, training deep learning models requires a large amount of labeled data, but it is difficult to obtain labeled data, and often requires a lot of manual labor. Secondly, the robustness of the algorithm still needs to be further improved. For example, the algorithm may be interfered by environmental factors such as light and occlusion. In addition, the generalization ability of the algorithm needs to be further improved. For example, the accuracy of the algorithm may vary in different groups of people and different scenarios. Due to the complexity of deep learning algorithms, their computational complexity and memory usage are high, and the algorithm is not very adaptable to resource-constrained devices such as embedded systems. Finally, although the current fatigue detection method based on the neural network model can achieve a good detection effect, the weight file obtained by training the model is large, the model has huge parameters and calculations, and it takes a long time, which is not suitable for running on mobile terminals and embedded devices. The driver's face needs to be detected in real time, and a lightweight algorithm model is needed for application. Summary of the invention

[0007] The purpose of the present invention is to provide a lightweight fatigue driving detection method based on the fusion of ECLA-HSFPN to solve the problems raised in the above background technology.

[0008] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0009] A lightweight fatigue driving detection method integrating ECLA-HSFPN, the method comprising:

[0010] Collect fatigue driving images, annotate them using Labeling tools, and create relevant data sets;

[0011] Build a network model that integrates yolov8 and ECLA-HSFPN;

[0012] Setting parameters for the improved network model and conducting training;

[0013] Build a fatigue driving model and evaluate the model performance on the test set.

[0014] Preferably, fatigue driving images are collected and annotated using Labeling tools to create relevant data sets, including:

[0015] Set three driving states: normal driving state, mild fatigue state and severe fatigue state;

[0016] Using cameras to collect video data shot by each user in each driving state;

[0017] The detection frequency of the camera is 30 frames per second;

[0018] The collected video data is extracted frame by frame, and the image data of each user in three driving states is randomly extracted; the face in each randomly extracted picture is labeled using the dataset labeling tool Labeling. The labeling information is the diagonal pixel coordinates of the bounding box and the labeling category, and is saved as an .xml format file. The dataset format produced is VOC format, and then the dataset is divided into training set, validation set, and test set in a ratio of 8:1:1 to complete the dataset production.

[0019] Preferably, a network model integrating yolov8 and ECLA-HSFPN is constructed, including:

[0020] A lightweight detection model is deployed using the single-stage YOLO series algorithm as the benchmark model, and the dataset is input into the improved YOLOv8 network model to start training;

[0021] The network model includes: a backbone feature extraction network, a neck feature fusion network and three detection heads;

[0022] The backbone feature extraction network is used to extract features;

[0023] The neck feature fusion network is used to collect image features of different scales and perform multi-scale feature fusion;

[0024] The three detection heads are used to predict bounding boxes and categories based on feature maps. After training, a weight file is obtained and the training is completed.

[0025] Preferably, the neck feature fusion network is used to aggregate image features of different scales and perform multi-scale feature fusion, including:

[0026] The neck feature fusion network uses ECLA-HSFPN to achieve multi-scale feature fusion, which is improved based on HS-FPN; the HS-FPN is used to filter low-level feature information using high-level features as weights, and then merge the filtered information with high-level features to enhance the feature expression ability of the model;

[0027] The HS-FPN includes a feature selection module and a feature fusion module;

[0028] The feature selection module is used to optimize features at different scales, reduce unnecessary information, and select representative features;

[0029] The feature selection module includes global average pooling and global maximum pooling; the global average pooling is used to add the output results and then apply the sigmoid function to determine the weight of each channel, and extract representative features from each channel; the global maximum pooling is used to multiply the obtained weight information with the corresponding scale feature map to optimize the feature validity;

[0030] Based on the improvement of HS-FPN, a spatial feature self-filtering module is added to the feature selection module;

[0031] The spatial feature self-filtering module is used to effectively utilize spatial information without sacrificing the channel dimension. First, a strip pooling strategy is adopted in the spatial dimension to extract horizontal and vertical features in the spatial dimension, and a narrow pooling kernel is used to capture the spatial mutuality of long-distance dependence, while filtering out information that is useless for the prediction target, and extracting directional position features. Then, by applying one-dimensional convolution to the feature vectors in two directions, the convolution kernel size is flexibly adjusted to control the range of local feature interaction and enhance the directional expression ability of the feature. The operated feature vectors are then processed by group normalization and nonlinear activation function to be converted into position attention weight predictions representing the two directions. Finally, the attention area is accurately located by multiplying the attention predictions in the two directions.

[0032] The feature fusion module uses high-level features as weights to strategically fuse features. Given the input high-level features and low-level features, the dimensions of the high-level features and low-level features are first unified using transposed convolution and bilinear interpolation, and then the channel feature self-filtering module is applied to convert the high-level features into corresponding weights for filtering low-level features, and finally the filtered low-level features are fused with the high-level features.

[0033] By adding a feature self-filtering module, the ECLA module is formed together with the original feature selection module, and the combination of transposed convolution and bilinear difference is replaced by a dynamic upsampler to obtain the neck feature fusion network.

[0034] Preferably, the dynamic upsampler comprises:

[0035] The dynamic upsampler is used to bypass the convolution operation and construct a dynamic upsampling process from the perspective of point sampling;

[0036] First, the feature map is input into the fully connected layer, and the output is a grid O consisting of offset vectors, which guide the precise offset of each pixel when it is upsampled;

[0037] Secondly, a number of offset vectors are fused with the original sampling grid G ​​to define a set of novel sampling coordinates S to guide the extraction of unconventional positions of pixels from the low-resolution image; wherein G is a regular grid representing the corresponding position of each pixel in the original image when upsampling according to conventional bilinear interpolation;

[0038] Finally, using the specific coordinates in S, the grid_sample function is used to generate Figure X Sampling is performed on the , generating the required high resolution output X′.

[0039] Preferably, setting parameters for the improved network model and training it includes:

[0040] Use the detection model to train the parameters of the network model;

[0041] The detection model uses the KAN network to achieve nonlinear changes. The KAN network is a new type of neural network architecture. It has no linear combination operation. It directly activates the nonlinear transformation of the input and then adds it at the node. It is equivalent to placing a learnable activation function on the edge of the network instead of on the node. The node only performs the addition operation.

[0042] The KAN network, according to the KAN network formula Get the KAN network value f(x); where, represents the sum of the internal functions, φ q,p Represents an internal function, represents the sum of the external functions, Φ q Represents an outer function that combines the outputs of inner functions to produce the final output.

[0043] A computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps in the above-mentioned lightweight fatigue driving detection method integrating ECLA-HSFPN are implemented.

[0044] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the above-mentioned lightweight fatigue driving detection method integrating ECLA-HSFPN are implemented.

[0045] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0046] Compared with the yolov8 version suitable for vehicle-mounted fatigue driving detection portable equipment, the present invention has improved accuracy and greatly reduced network parameters and model size;

[0047] 1. The present invention achieves lightweight: The present invention proposes a lightweight image detection method and system integrating ECLA-HSFPN, which is deeply customized and optimized based on YOLOv8. In the Head network, ECLA-HSFPN is used to realize multi-scale feature fusion, and high-level features are used as weights to filter low-level feature information. The filtered information is then merged with high-level features to enhance the feature expression ability of the model, so that it has more flexible feature selectivity and stronger feature expression ability, thereby further improving the performance of the model in processing target detection tasks. The network can process targets of different sizes more effectively and greatly reduce the number of model parameters. Image target feature extraction technology is used for prediction to identify images, so that the overall accuracy stability and generalization ability of the model detection are improved to a certain extent, and the number of parameters and model size are significantly reduced, thereby achieving the lightweight demand.

[0048] 2. High computational efficiency: The present invention is improved on the YOLOv8 framework, while taking into account the particularity of the image processing process, and adaptively improves each module. Experimental verification shows that while maintaining the same detection accuracy, the number of model parameters and the amount of computation can be significantly reduced, thereby greatly improving computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0050] Figure 1 It is a flow chart of a lightweight fatigue driving detection method integrating ECLA-HSFPN of the present invention;

[0051] Figure 2 This is the overall structure diagram of the network model of ECLA-HSFPN fusion of the present invention;

[0052] Figure 3 It is a working flow chart of the HSFPN module of the present invention;

[0053] Figure 4 It is a schematic diagram of the image processing process of the HSFPN module of the present invention;

[0054] Figure 5 is a schematic diagram of the ECLA module of the present invention;

[0055] Figure 6 It is a schematic diagram of the spatial feature self-filtering module of the present invention;

[0056] Figure 7 It is a schematic diagram of the feature selection module of the present invention;

[0057] Figure 8 is a schematic diagram of the improved feature fusion module of the present invention;

[0058] Fig. 9 is a work flow chart of the improved feature fusion module of the present invention;

[0059] Fig.10 It is a curve diagram of the network model training process proposed by the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] See also Figure 1-Figure 10 , the present invention provides a technical solution:

[0062] Embodiment 1:

[0063] A lightweight fatigue driving detection method integrating ECLA-HSFPN, the method comprising:

[0064] S1. Collect fatigue driving images and annotate them using Labeling tools to create relevant data sets;

[0065] S2. Build a network model that integrates yolov8 and ECLA-HSFPN;

[0066] S3, setting parameters for the improved network model and training it;

[0067] S4. Build a fatigue driving model and evaluate the model performance on the test set.

[0068] Preferably, S1, collect fatigue driving images, and use Labeling tools to annotate them to create relevant data sets, including:

[0069] Set three driving states: normal driving state, mild fatigue state and severe fatigue state;

[0070] Construction of the dataset: Targeting severe fatigue status:

[0071] Before data collection, the three volunteers were asked to stay up all night and remain in a state of severe fatigue. When preparing for data collection, in order to ensure road traffic safety, the driver in a sober state was asked to drive the car to a safe position and stop the car. Then a camera was placed in front of the steering wheel to record fatigue driving information to ensure that the driver's line of sight was not blocked. Afterwards, the three volunteers in a state of severe fatigue took turns to sit in the driving seat to simulate a fatigue driving state. During this period, the driver could speak freely, observe, etc., to simulate the real driving state as much as possible. All video data of the volunteers in the process of simulating fatigue driving were collected, and in the subsequent data sorting process, the changes in the volunteers' eyes when they were severely fatigued and the frequent yawning behavior were distinguished;

[0072] For normal driving conditions:

[0073] A few days after the severe fatigue state was collected, the three volunteers recovered their mental state and drove the car to a safe parking position and stopped the engine to ensure traffic safety. Then they placed a camera at the same location to prepare for data collection. During data collection, the three volunteers in good mental state also entered the cab one by one, simulated the driving state, and recorded the normal driving process simulated by the volunteers through the camera. When processing the data later, the normal driving state of each driver was sorted out separately;

[0074] For mild fatigue:

[0075] The three volunteers were asked to perform the above-mentioned filming process in the hungry state before lunch and in the digestive state for a period of time after the meal to obtain data on the mild fatigue state of the drivers;

[0076] Using cameras to collect video data shot by each user in each driving state;

[0077] The detection frequency of the camera is 30 frames per second;

[0078] The detection frequency of the camera is 30 frames per second. The fatigue driving experiment test data produced includes 3 groups of normal driving videos, 3 groups of mild fatigue driving videos, and 3 groups of severe fatigue driving videos, in preparation for subsequent experiments and tests.

[0079] The collected video data is extracted frame by frame, and the image data of each user in three driving states is randomly extracted; the face in each randomly extracted picture is labeled using the dataset labeling tool Labeling. The labeling information is the diagonal pixel coordinates of the bounding box and the labeling category, and is saved as an .xml format file. The dataset format produced is VOC format, and then the dataset is divided into training set, validation set, and test set in a ratio of 8:1:1 to complete the dataset production.

[0080] Preferably, S2, building a network model integrating yolov8 and ECLA-HSFPN, including:

[0081] A lightweight detection model is deployed using the single-stage YOLO series algorithm as the benchmark model, and the dataset is input into the improved YOLOv8 network model to start training;

[0082] The network model includes: a backbone feature extraction network, a neck feature fusion network and three detection heads;

[0083] To ensure that the lightweight improvement meets the needs, the classic YOLOv8 algorithm is selected, and the backbone feature extraction network is used to extract features;

[0084] The neck feature fusion network is used to collect image features of different scales and perform multi-scale feature fusion;

[0085] The three detection heads are used to predict bounding boxes and categories based on feature maps. After training, a weight file is obtained and the training is completed.

[0086] Preferably, the neck feature fusion network is used to aggregate image features of different scales and perform multi-scale feature fusion, including:

[0087] The neck feature fusion network uses ECLA-HSFPN to achieve multi-scale feature fusion, which is improved based on HS-FPN; the HS-FPN is used to filter low-level feature information using high-level features as weights, and then merge the filtered information with high-level features to enhance the feature expression ability of the model;

[0088] The HS-FPN includes a feature selection module and a feature fusion module;

[0089] The feature selection module is used to optimize features at different scales, reduce unnecessary information, and select representative features;

[0090] The feature selection module includes global average pooling and global maximum pooling; the global average pooling is used to add the output results and then apply the sigmoid function to determine the weight of each channel, and extract representative features from each channel; the global maximum pooling is used to multiply the obtained weight information with the corresponding scale feature map to optimize the feature validity;

[0091] Based on the improvement of HS-FPN, a spatial feature self-filtering module is added to the feature selection module;

[0092] The spatial feature self-filtering module is used to effectively utilize spatial information without sacrificing the channel dimension. First, a strip pooling strategy is adopted in the spatial dimension to extract horizontal and vertical features in the spatial dimension, and a narrow pooling kernel is used to capture the spatial mutuality of long-distance dependence, while filtering out information that is useless for the prediction target, and extracting directional position features. Then, by applying one-dimensional convolution to the feature vectors in two directions, the convolution kernel size is flexibly adjusted to control the range of local feature interaction and enhance the directional expression ability of the feature. The operated feature vectors are then processed by group normalization and nonlinear activation function to be converted into position attention weight predictions representing the two directions. Finally, the attention area is accurately located by multiplying the attention predictions in the two directions.

[0093] The original feature selection module is combined with the ELA spatial feature self-filtering module and is called ECLA, which can effectively utilize spatial information, reduce the overall computational workload of the model, and improve computational efficiency;

[0094] The feature fusion module uses high-level features as weights to strategically fuse features. Given the input high-level features and low-level features, the dimensions of the high-level features and low-level features are first unified using transposed convolution and bilinear interpolation, and then the channel feature self-filtering module is applied to convert the high-level features into corresponding weights for filtering low-level features, and finally the filtered low-level features are fused with the high-level features.

[0095] By adding a feature self-filtering module, together with the original feature selection module, the ECLA module is constructed, and the combination of transposed convolution and bilinear difference is replaced by a dynamic upsampler to obtain a neck feature fusion network.

[0096] The present invention replaces the channel feature self-filtering module with the ECLA module, and then replaces the combination of transposed convolution and bilinear interpolation with a dynamic upsampler. Bilinear interpolation applies fixed rules to interpolate low-resolution features, cannot adapt to the specificity of the data set, and ignores the semantic information in the feature map. Transposed convolution involves more calculations and has a higher computational complexity.

[0097] Preferably, the dynamic upsampler comprises:

[0098] The dynamic upsampler is used to bypass the convolution operation and construct a dynamic upsampling process from the perspective of point sampling;

[0099] First, in order to dynamically determine the location of the sampling point, the feature map is input into the fully connected layer, and the output is a grid O consisting of offset vectors, which guides the precise offset of each pixel when it is upsampled;

[0100] Secondly, a number of offset vectors are fused with the original sampling grid G ​​to define a set of novel sampling coordinates S to guide the extraction of unconventional positions of pixels from the low-resolution image; wherein G is a regular grid representing the corresponding position of each pixel in the original image when upsampling according to conventional bilinear interpolation;

[0101] Finally, using the specific coordinates in S, the grid_sample function is used to generate Figure X Sampling is performed on the , generating the required high resolution output X′.

[0102] Preferably, S3, setting parameters for the improved network model and training it, includes:

[0103] Use the detection model to train the parameters of the network model;

[0104] The detection model is based on a multi-layer perceptron. The essence of the multi-layer perceptron is to use a nonlinear activation function to realize nonlinear changes on the linear combination of inputs. Its advantages are simplicity and effectiveness. However, MLP itself has limitations, such as gradient vanishing and gradient explosion in back propagation gradient calculation; low parameter efficiency; limited ability to process high-dimensional data; difficulty in capturing long-term dependencies in input sequences, etc.

[0105] Therefore, the detection model of the present invention uses a KAN network to achieve nonlinear changes. The KAN network is a new type of neural network architecture. It has no linear combination operation, directly activates nonlinear transformation on the input, and then adds it at the node, which is equivalent to placing a learnable activation function on the edge of the network instead of on the node. The node only performs addition operation;

[0106] The KAN network, according to the KAN network formula Get the KAN network value f(x); where, represents the sum of the internal functions, φ q,p Represents an internal function, represents the sum of the external functions, Φ q represents the outer function, which combines the outputs of the inner functions to generate the final output;

[0107] This structure allows KAN to approximate complex multivariate functions in a highly flexible and expressive way;

[0108] Based on the network model of YOLOv8, the Bottleneck module of C2f in the Neck module of the YOLOv8 network model is obtained. The present invention replaces the Conv layer in the Bottleneck module with a KAN network. Bottleneck is the bottleneck part for improving the performance of the model. If this part is optimized, it will directly lead to the improvement of the overall performance. The Bottleneck module is usually used to reduce the dimension of the feature map and then restore it to the original dimension.

[0109] This design helps to reduce the amount of computation and increase the depth of the network. The KAN network can effectively learn low-dimensional feature representations. Using the KAN network in the Bottleneck module will improve the efficiency and performance of the entire network. The Bottleneck module achieves feature fusion through residual connections, which helps to retain more detailed information. The KAN network can better capture and fuse these features due to its learnable activation function. The Bottleneck module helps the network learn more generalized features by reducing the number of channels and then increasing them back. The learnable activation function of the KAN network further enhances this generalization ability. Compared with the Conv layer, the activation function of the KAN network is learnable and can be adaptively adjusted to better capture the characteristics of the input data; the learnable activation function helps to reduce overfitting, especially when the training samples are limited or the categories are unbalanced.

[0110] Preferably, based on S4, a fatigue driving model is constructed, and the model performance is evaluated on a test set, and the experimental data of the performance of the product or result is obtained as shown in the following table:

[0111]

[0112]

[0113] Embodiment 2:

[0114] The computer-readable storage medium of this embodiment stores a computer program thereon, and when the program is executed by a processor, the steps in a lightweight fatigue driving detection method integrating ECLA-HSFPN in embodiment 1 are implemented.

[0115] The computer-readable storage medium of this embodiment may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal; the computer-readable storage medium of this embodiment may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, etc. equipped on the terminal; further, the computer-readable storage medium may also include both an internal storage unit of the terminal and an external storage device.

[0116] The computer-readable storage medium of this embodiment is used to store computer programs and other programs and data required by the terminal. The computer-readable storage medium can also be used to temporarily store data that has been output or is to be output.

[0117] Embodiment 3:

[0118] The computer device of this embodiment includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in a lightweight fatigue driving detection method integrating ECLA-HSFPN in Embodiment 1 are implemented.

[0119] In this embodiment, the processor may be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, readily available programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0120] Those skilled in the art will appreciate that the disclosed content of the embodiments may be provided as methods, systems, or computer program products. Therefore, the present solution may adopt the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Moreover, the present solution may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program codes.

[0121] The present solution is described with reference to the method according to the embodiment of the present solution and the flowchart and / or block diagram of the computer program product. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions; these computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 one or more processes and / or methods Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0122] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 one or more processes and / or methods Figure 1 A function specified in one or more boxes.

[0123] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 one or more processes and / or methods Figure 1 A step that specifies a function in one or more boxes.

[0124] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0125] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A lightweight fatigue driving detection method integrating ECLA-HSFPN, characterized by: The method comprises: Collect fatigue driving images, annotate them using Labeling tools, and create relevant data sets; Build a network model that integrates yolov8 and ECLA-HSFPN; Setting parameters for the improved network model and conducting training; Build a fatigue driving model and evaluate the model performance on the test set.

2. A lightweight fatigue driving detection method integrating ECLA-HSFPN as claimed in claim 1, characterized in that: The fatigue driving images are collected and annotated using the Labeling tool to create relevant data sets, including: Set three driving states: normal driving state, mild fatigue state and severe fatigue state; Using cameras to collect video data shot by each user in each driving state; The detection frequency of the camera is 30 frames per second; The collected video data is extracted frame by frame, and the image data of each user in three driving states is randomly extracted; the face in each randomly extracted picture is labeled using the dataset labeling tool Labeling. The labeling information is the diagonal pixel coordinates of the bounding box and the labeling category, and is saved as an .xml format file. The dataset format produced is VOC format, and then the dataset is divided into training set, validation set, and test set in a ratio of 8:1:1 to complete the dataset production.

3. A lightweight fatigue driving detection method integrating ECLA-HSFPN as claimed in claim 1, characterized in that: The network model of building the fusion of yolov8 and ECLA-HSFPN includes: A lightweight detection model is deployed using the single-stage YOLO series algorithm as the benchmark model, and the dataset is input into the improved YOLOv8 network model to start training; The network model includes: a backbone feature extraction network, a neck feature fusion network and three detection heads; The backbone feature extraction network is used to extract features; The neck feature fusion network is used to collect image features of different scales and perform multi-scale feature fusion; The three detection heads are used to predict bounding boxes and categories based on feature maps. After training, a weight file is obtained and the training is completed.

4. A lightweight fatigue driving detection method integrating ECLA-HSFPN as claimed in claim 3, characterized in that: The neck feature fusion network is used to collect image features of different scales and perform multi-scale feature fusion, including: The neck feature fusion network uses ECLA-HSFPN to achieve multi-scale feature fusion, which is improved based on HS-FPN; the HS-FPN is used to filter low-level feature information using high-level features as weights, and then merge the filtered information with high-level features to enhance the feature expression ability of the model; The HS-FPN includes a feature selection module and a feature fusion module; The feature selection module is used to optimize features at different scales, reduce unnecessary information, and select representative features; The feature selection module includes global average pooling and global maximum pooling; the global average pooling is used to add the output results and then apply the sigmoid function to determine the weight of each channel, and extract representative features from each channel; the global maximum pooling is used to multiply the obtained weight information with the corresponding scale feature map to optimize the feature validity; Based on the improvement of HS-FPN, a spatial feature self-filtering module is added to the feature selection module; The spatial feature self-filtering module is used to effectively utilize spatial information without sacrificing the channel dimension. First, a strip pooling strategy is adopted in the spatial dimension to extract horizontal and vertical features in the spatial dimension, and a narrow pooling kernel is used to capture the spatial mutuality of long-distance dependence, while filtering out information that is useless for the prediction target, and extracting directional position features. Then, by applying one-dimensional convolution to the feature vectors in two directions, the convolution kernel size is flexibly adjusted to control the range of local feature interaction and enhance the directional expression ability of the feature. The operated feature vectors are then processed by group normalization and nonlinear activation function to be converted into position attention weight predictions representing the two directions. Finally, the attention area is accurately located by multiplying the attention predictions in the two directions. The feature fusion module uses high-level features as weights to strategically fuse features. Given the input high-level features and low-level features, the dimensions of the high-level features and low-level features are first unified using transposed convolution and bilinear interpolation, and then the channel feature self-filtering module is applied to convert the high-level features into corresponding weights for filtering low-level features, and finally the filtered low-level features are fused with the high-level features. By adding a feature self-filtering module, the ECLA module is formed together with the original feature selection module, and the combination of transposed convolution and bilinear difference is replaced by a dynamic upsampler to obtain the neck feature fusion network.

5. A lightweight fatigue driving detection method integrating ECLA-HSFPN as claimed in claim 4, characterized in that: The dynamic upsampler comprises: The dynamic upsampler is used to bypass the convolution operation and construct a dynamic upsampling process from the perspective of point sampling; First, the feature map is input into the fully connected layer, and the output is a grid O consisting of offset vectors, which guide the precise offset of each pixel when it is upsampled; Secondly, a number of offset vectors are fused with the original sampling grid G ​​to define a set of novel sampling coordinates S to guide the extraction of unconventional positions of pixels from the low-resolution image; wherein G is a regular grid representing the corresponding position of each pixel in the original image when upsampling according to conventional bilinear interpolation; Finally, using the specific coordinates in S, the grid_sample function is used to perform sampling on the original image X to generate the required high-resolution output X′.

6. A lightweight fatigue driving detection method integrating ECLA-HSFPN as claimed in claim 1, characterized in that: The step of setting parameters for the improved network model and training the model includes: Use the detection model to train the parameters of the network model; The detection model uses the KAN network to achieve nonlinear changes. The KAN network is a new type of neural network architecture. It has no linear combination operation. It directly activates the nonlinear transformation of the input and then adds it at the node. It is equivalent to placing a learnable activation function on the edge of the network instead of on the node. The node only performs the addition operation. The KAN network, according to the KAN network formula Get the KAN network value f ( x); where represents the sum of the internal functions, φ q,p Represents an internal function, represents the sum of the external functions, Φ q Represents an outer function that combines the outputs of inner functions to produce the final output.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in a lightweight fatigue driving detection method integrating ECLA-HSFPN as described in any one of claims 1 to 6 are implemented.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in a lightweight fatigue driving detection method integrating ECLA-HSFPN as described in any one of claims 1-7 are implemented.

Images