Crane fatigue driving detection method based on improved YOLO

Abstract

The application provides a crane fatigue driving detection method based on improved YOLO, relates to the technical field of fatigue driving detection, and comprises the following steps: collecting video data of a driver through a camera device, extracting frame images, and forming an image sequence; performing image preprocessing on the image sequence; obtaining final feature parameters of each facial key target of the driver through an improved YOLO network; constructing a unified feature analysis model based on the numerical values of the final feature parameters of all facial key targets of the image sequence and the attributes thereof, and the unified feature analysis model is used for feature extraction on all facial key targets; and judging whether the driver is fatigued driving according to the result of feature extraction. The application realizes unified analysis of different facial key target features and improves the universality and scalability of the fatigue driving detection method.

Description

Technical Field

[0001] This invention relates to the field of fatigue driving detection technology, and more specifically, to a method for detecting fatigue driving of cranes based on an improved YOLO. Background Technology

[0002] Crane operators are often in high-intensity working environments and are prone to fatigue due to prolonged continuous work.

[0003] With the development of technologies such as artificial intelligence, existing technologies have begun to attempt driver fatigue detection based on visual image analysis, focusing on fatigue detection based on various key facial features. However, the feature parameters corresponding to different key facial features have different physical meanings and variation patterns. This approach requires the analysis and fusion of multiple facial features one by one, resulting in a complex algorithm structure. Moreover, if new key facial features emerge with advancements in detection technology, corresponding analysis modules need to be added simultaneously, leading to poor algorithm scalability and versatility.

[0004] Therefore, it is urgent to optimize image analysis-based fatigue driving detection methods to achieve unified analysis of different key facial features and improve the versatility and scalability of fatigue driving detection methods. Summary of the Invention

[0005] The purpose of this invention is to provide a crane fatigue driving detection method based on an improved YOLO, which achieves unified analysis of different key facial features.

[0006] This invention is achieved through the following technical solution: The crane fatigue driving detection method based on the improved YOLO includes the following steps: The driver's video data is collected by a camera device, and N frames of images up to the current time are extracted from the video data at a preset frequency to form an image sequence; Image preprocessing is performed on the image sequence; Based on the image sequence, the final feature parameters of each key facial target of the driver are obtained by improving the YOLO network. The final feature parameters of the key facial target include the normalized value of the difference between the maximum and minimum row index of all its pixels, and the normalized value of the difference between the maximum and minimum column index. Based on the final feature parameters and attributes of all key facial targets in the image sequence, a unified feature analysis model is constructed. The attributes are positive or negative indicators. The unified feature analysis model is used to extract features for all key facial targets. The driver's fatigue can be determined based on the results of feature extraction.

[0007] Preferably, the camera device is installed inside the cockpit and directly facing the driver's face.

[0008] Preferably, the method for image preprocessing the image sequence includes: Perform gamma correction; Image enhancement is achieved by limiting contrast-adaptive histogram equalization.

[0009] Preferably, the improved YOLO network includes: In the backbone layer, a multi-scale attention module is set at the output of the CSP / Conv module. The multi-scale attention module is used to calculate channel attention and spatial attention. The output of the CSP / Conv module is corrected by channel attention and spatial attention in turn to obtain the spatial attention enhanced feature map. The multi-scale fusion layer uses a bidirectional weighted feature pyramid network to fuse spatial attention-enhanced feature maps to obtain fused features. The decoupled detection head includes a common convolutional network, a localization branch, a classification branch, and a key point branch. The localization branch, classification branch, and key point branch are all connected to the output of the common convolutional network. The common convolutional network is used to perform convolution processing on the fused features. The localization branch is used to locate the face, the classification branch is used to identify key facial targets, and the key point branch is used to extract key feature points for identifying key facial targets.

[0010] Preferably, the spatial attention enhancement feature map is calculated. The method is as follows: Calculate the channel attention Channel attention enhancement feature map :

[0011] ; in, For the sigmoid function, This is the global average pooling function. The global max pooling function is MLP, which is a fully connected layer, and F is the image feature output by the CSP / Conv module. Calculate the spatial attention and the spatial attention-enhanced feature map. : ; ; in, The kernel size represents the convolution kernel size. The convolutional layer.

[0012] Preferably, both the localization branch and the classification branch adopt a regression branch structure, including multiple convolutional layers in series, and are trained with corresponding parameters according to their respective recognition targets; The calculation method for the key point branches is as follows: Obtain the key points of all the facial key targets respectively. The key points include the point with the largest row index, the point with the smallest row index, the point with the largest column index, and the point with the smallest column index of the facial key targets in the image. Calculate the difference between the maximum and minimum row indexes of the facial key target, as well as the difference between the maximum and minimum column indexes, and then normalize them.

[0013] Preferably, the key facial targets include the left eye, right eye, and mouth.

[0014] Preferably, the method for feature extraction based on the final feature parameters of all key facial targets in the image sequence is as follows: Calculate the average evaluation parameter and average change parameter of the final feature parameters for each of the facial key targets in the image sequence: ; ; in, and The first The average evaluation parameter and average change parameter of the final feature parameters of the aforementioned key facial targets. The first of the image sequence The first frame The normalized value of the difference between the maximum and minimum row indices of the aforementioned key facial targets. The image sequence is respectively the first The first frame The normalized value of the difference between the maximum and minimum column indices of the facial key target. The extraction time difference between two adjacent frame images; Construct state evaluation parameters for each of the aforementioned key facial targets: ; in, For the first The state evaluation parameters for the aforementioned key facial targets. and The weights are 1, and their sum is 1. For the first The mapping function of the average evaluation parameter of the final feature parameters of the facial key target is described, and the mapped value is a negative index.

[0015] Preferably, if the As a negative indicator, the mapping function The output is If the stated As a positive indicator, the mapping function The output is .

[0016] Preferably, the method for determining whether a driver is driving while fatigued based on the results of the feature extraction is as follows: Obtain the maximum value of the status evaluation parameters of all the key facial targets. If the maximum value exceeds the preset threshold, it is determined that the driver is driving while fatigued.

[0017] The technical solution of the present invention has at least the following advantages and beneficial effects: This invention obtains the final feature parameters of each key facial target of the driver by improving the YOLO network, and uses a unified form to describe the spatial changes of each key facial target, providing a basis for subsequent feature analysis in the same way. This invention performs unified feature extraction based on the final feature parameters and attributes of each key facial target, without the need to establish different analysis models for different key facial targets such as eyes and mouth, thus simplifying the fatigue driving detection process. In the feature extraction stage, this invention only uses the final feature parameter values ​​and their corresponding indicator attributes for analysis, without relying on the information of the location of the key facial target. When a new key facial target is added, it is only necessary to determine the final feature parameter values ​​and their corresponding indicator attributes to connect to the unified analysis process, without having to redesign a new feature analysis algorithm, thus improving the system's scalability. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the crane fatigue driving detection method based on the improved YOLO provided in Embodiment 1 of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0020] Example 1 This embodiment provides a crane fatigue driving detection method based on an improved YOLO, see reference. Figure 1 This includes the following steps: Step S1: Set the camera device in the cockpit, facing the driver's face, and then collect the driver's video data through the camera device. Extract N frames of images up to the current time from the video data according to a preset frequency to form an image sequence.

[0021] Step S2: To reduce the impact of changes in lighting inside the crane cab and insufficient contrast in the operator's facial area on image data processing, image preprocessing is required for the image sequence. Methods include: Gamma correction is performed to adjust the overall brightness distribution of an image. Gamma correction methods typically involve adjusting the pixel grayscale values ​​of the image. The corrected pixel grayscale value is obtained by multiplying the power by the grayscale gain coefficient. It is the gamma correction parameter; Image enhancement is achieved by limiting contrast adaptive histogram equalization (CLAHE), which improves the grayscale difference between key facial regions and the surrounding background.

[0022] Step S3: Based on the image sequence, the final feature parameters of each key facial feature of the driver are obtained by improving the YOLO network. The final feature parameters of the key facial feature include the normalized value of the difference between the maximum and minimum row indices of all its pixels, and the normalized value of the difference between the maximum and minimum column indices. In one implementation, the key facial features include the left eye, right eye, and mouth.

[0023] The improved YOLO network in this embodiment includes: In the backbone layer, to improve feature representation capabilities, a multi-scale attention module is set at the output of the CSP / Conv module. The multi-scale attention module is used to calculate channel attention and spatial attention. The output of the CSP / Conv module is corrected sequentially through channel attention and spatial attention to obtain a spatial attention-enhanced feature map. On the one hand, channel attention is used to evaluate the importance of different feature channels. This reflects the weights of each channel. Multiplying by the principal component yields the attention-enhanced image after weight correction for each channel. On the other hand, spatial attention can enhance the extraction of key facial targets.

[0024] As a preferred approach, the spatial attention enhancement feature map is calculated. The method is as follows: Calculate the channel attention Channel attention enhancement feature map :

[0025] ; in, For the sigmoid function, This is the global average pooling function. The global max pooling function is MLP, which is a fully connected layer, and F is the image feature output by the CSP / Conv module. Calculate the spatial attention and the spatial attention-enhanced feature map. : ; ; in, The kernel size represents the convolution kernel size. The convolutional layer.

[0026] The multi-scale fusion layer employs a bidirectional weighted feature pyramid network (BiFPN) to fuse spatial attention-enhanced feature maps, resulting in fused features. The main working principle of BiFPN is to establish bidirectional information propagation paths from top to bottom and bottom to top, assigning learnable fusion weights to features at different scales. This achieves a full fusion of semantic and detail information at different scales. Using this network enables better detection results for different facial key targets with significant size differences, improving detection robustness under complex conditions.

[0027] The decoupled detection head includes a common convolutional network, a localization branch, a classification branch, and a key point branch. The localization branch, classification branch, and key point branch are all connected to the output of the common convolutional network. The common convolutional network is used to perform convolution processing on the fused features. The localization branch is used to locate the face, the classification branch is used to identify key facial targets, and the key point branch is used to extract key feature points for identifying key facial targets.

[0028] In this embodiment, both the localization branch and the classification branch adopt a regression branch structure. The regression branch structure typically includes multiple convolutional layers connected in series. In the construction of the localization branch and the classification branch, corresponding parameters are trained according to their respective recognition targets. The calculation method for the key point branches is as follows: Obtain the key points of all the facial key targets respectively. The key points include the point with the largest row index, the point with the smallest row index, the point with the largest column index, and the point with the smallest column index of the facial key targets in the image. Calculate the difference between the maximum and minimum row indexes of the facial key target, and the difference between the maximum and minimum column indexes, and perform normalization processing. The preferred normalization processing method is: divide the difference between the maximum and minimum column indexes by the total number of column indices of the image, and divide the difference between the maximum and minimum row indexes by the total number of row indices. Here, the indexes can be based on the target detection boxes of the facial key targets identified by the YOLO network.

[0029] Because the essential changes in all key facial targets during fatigue detection are alterations in size and shape, this scheme uses extreme value index differences to describe the spatial distribution of key facial targets. It utilizes the row and column index changes of each target to uniformly represent the morphological changes of different key facial targets, eliminating the need to design separate geometric feature calculation models for the left eye, right eye, and mouth. Subsequent unified feature analysis can be completed simply by combining the index attributes corresponding to each final feature parameter, thereby reducing the algorithmic complexity of separately modeling and fusing different key facial targets, while improving the system's versatility and scalability. For example, features such as eyebrows (which deform when yawning) and cheek contours (which change shape when the facial surface contour is captured by posture changes) can be added. The feature data processing of these key facial targets can all adopt the unified processing method of this embodiment, making expansion simple.

[0030] Step S4: Based on the final feature parameter values ​​and attributes of all key facial targets in the image sequence, construct a unified feature analysis model. The attributes can be positive or negative indicators. This unified feature analysis model is used for feature extraction from all key facial targets. A preferred implementation method is as follows: Calculate the average evaluation parameter and average change parameter of the final feature parameters for each of the facial key targets in the image sequence: ; ; in, and The first The average evaluation parameter and average change parameter of the final feature parameters of the aforementioned key facial targets. The first of the image sequence The first frame The normalized value of the difference between the maximum and minimum row indices of the aforementioned key facial targets. The image sequence is respectively the first The first frame The normalized value of the difference between the maximum and minimum column indices of the facial key target. The extraction time difference between two adjacent frame images; Construct state evaluation parameters for each of the aforementioned key facial targets: ; in, For the first The state evaluation parameters for the aforementioned key facial targets. and The weights are 1, and their sum is 1. For the first The mapping function is used to evaluate the average value of the final feature parameters of the aforementioned key facial targets, and the mapped value is a negative indicator. Specifically, if the... As a negative indicator, the mapping function The output is If the stated As a positive indicator, the mapping function The output is Positive indicators indicate that the higher the final characteristic parameter value, the lower the driver's fatigue level (the better the situation); negative indicators indicate that the lower the final characteristic parameter value, the higher the driver's fatigue level (the worse the situation). For example, when fatigued, the characteristics are usually that the eyes shrink (drowsiness) and the mouth widens (yawning), so for the eyes... The smaller the value, the more likely it is to indicate fatigue, especially for the mouth. A higher number indicates potential fatigue. And regardless of the specific facial target, it is... The larger the size (i.e., the more frequently the state changes), the more likely fatigue is to occur. The higher the value, the worse the fatigue assessment.

[0031] Step S5: Determine whether the driver is driving while fatigued based on the feature extraction results. The method is as follows: Obtain the maximum value of the status evaluation parameters of all the key facial targets. If the maximum value exceeds the preset threshold, it is determined that the driver is driving while fatigued.

[0032] Since the safe range of values ​​for the state evaluation parameters of key facial targets varies across different scenarios, thresholds can be set through field experimental data collection. For example, the average value of the state evaluation parameters from multiple samples taken during normal driving can be collected. ,by The threshold is 80%-90% of the value.

[0033] This embodiment uses the maximum value among the state evaluation parameters of all key facial targets as the basis for fatigue determination, which can quickly capture the most significant fatigue features, sufficient for judgment. Furthermore, since all state evaluation parameters are calculated using a unified feature analysis model, when adding other key facial targets, only their indicator attributes need to be determined to directly integrate them into the above analysis process, without needing to redesign the corresponding feature extraction and evaluation algorithms, further improving the scalability of this invention.

[0034] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A crane fatigue driving detection method based on an improved YOLO, characterized in that, Includes the following steps: The driver's video data is collected by a camera device, and N frames of images up to the current time are extracted from the video data at a preset frequency to form an image sequence; Image preprocessing is performed on the image sequence; Based on the image sequence, the final feature parameters of each key facial target of the driver are obtained by improving the YOLO network. The final feature parameters of the key facial target include the normalized value of the difference between the maximum and minimum row index of all its pixels, and the normalized value of the difference between the maximum and minimum column index. Based on the final feature parameters and attributes of all key facial targets in the image sequence, a unified feature analysis model is constructed. The attributes are positive or negative indicators. The unified feature analysis model is used to extract features for all key facial targets. The driver's fatigue can be determined based on the results of feature extraction.

2. The crane fatigue driving detection method based on the improved YOLO according to claim 1, characterized in that, The camera device is placed inside the cockpit and directly facing the driver's face.

3. The crane fatigue driving detection method based on the improved YOLO according to claim 1, characterized in that, The method for image preprocessing the image sequence includes: Perform gamma correction; Image enhancement is achieved by limiting contrast-adaptive histogram equalization.

4. The crane fatigue driving detection method based on the improved YOLO according to claim 1, characterized in that, The improved YOLO network includes: In the backbone layer, a multi-scale attention module is set at the output of the CSP / Conv module. The multi-scale attention module is used to calculate channel attention and spatial attention. The output of the CSP / Conv module is corrected by channel attention and spatial attention in turn to obtain the spatial attention enhanced feature map. The multi-scale fusion layer uses a bidirectional weighted feature pyramid network to fuse spatial attention-enhanced feature maps to obtain fused features. The decoupled detection head includes a common convolutional network, a localization branch, a classification branch, and a key point branch. The localization branch, classification branch, and key point branch are all connected to the output of the common convolutional network. The common convolutional network is used to perform convolution processing on the fused features. The localization branch is used to locate the face, the classification branch is used to identify key facial targets, and the key point branch is used to extract key feature points for identifying key facial targets.

5. The crane fatigue driving detection method based on the improved YOLO according to claim 4, characterized in that, Calculate the spatial attention enhancement feature map The method is as follows: Calculate the channel attention Channel attention enhancement feature map : ； in, For the sigmoid function, This is the global average pooling function. The global max pooling function is MLP, which is a fully connected layer, and F is the image feature output by the CSP / Conv module. Calculate the spatial attention and the spatial attention-enhanced feature map. : ； ； in, The kernel size represents the convolution kernel size. The convolutional layer.

6. The crane fatigue driving detection method based on the improved YOLO according to claim 5, characterized in that, Both the localization branch and the classification branch adopt a regression branch structure, which includes multiple convolutional layers in series, and the corresponding parameters are trained according to their respective recognition targets. The calculation method for the key point branches is as follows: Obtain the key points of all the facial key targets respectively. The key points include the point with the largest row index, the point with the smallest row index, the point with the largest column index, and the point with the smallest column index of the facial key targets in the image. Calculate the difference between the maximum and minimum row indexes of the facial key target, as well as the difference between the maximum and minimum column indexes, and then normalize them.

7. The crane fatigue driving detection method based on the improved YOLO according to claim 6, characterized in that, The key facial features include the left eye, right eye, and mouth.

8. The crane fatigue driving detection method based on the improved YOLO according to claim 1, characterized in that, The method for extracting the features based on the final feature parameters of all key facial targets in the image sequence is as follows: Calculate the average evaluation parameter and average change parameter of the final feature parameters for each of the facial key targets in the image sequence: ； ； in, and The first The average evaluation parameter and average change parameter of the final feature parameters of the aforementioned key facial targets. The first of the image sequence The first frame The normalized value of the difference between the maximum and minimum row indices of the aforementioned key facial targets. The image sequence is respectively the first The first frame The normalized value of the difference between the maximum and minimum column indices of the facial key target. The extraction time difference between two adjacent frame images; Construct state evaluation parameters for each of the aforementioned key facial targets: ； in, For the first The state evaluation parameters for the aforementioned key facial targets. and The weights are 1, and their sum is 1. For the first The mapping function of the average evaluation parameter of the final feature parameters of the facial key target is described, and the mapped value is a negative index.

9. The crane fatigue driving detection method based on the improved YOLO according to claim 8, characterized in that, If the above As a negative indicator, the mapping function The output is If the stated As a positive indicator, the mapping function The output is .

10. The crane fatigue driving detection method based on the improved YOLO according to claim 9, characterized in that, The method for determining whether a driver is driving while fatigued based on the results of the feature extraction is as follows: Obtain the maximum value of the status evaluation parameters of all the key facial targets. If the maximum value exceeds the preset threshold, it is determined that the driver is driving while fatigued.

Images