Fatigue driving detection method and system

Abstract

The invention relates to a fatigue driving detection method and system, and the method comprises the steps: obtaining a distance range of the center of a human eye frame marked in a training sample in a video frame, dividing each frame of a shot driver video into a plurality of image blocks, fusing the image block and the position code of the image block, inputting the fused image block and the position code of the image block into an encoder to obtain a feature vector output by the encoder, and filtering the feature vector according to a feature vector clustering result, the position of the image block corresponding to the feature vector in the frame and the distance range; multiplying a feature matrix formed by the filtered feature vectors with transpose of a matrix formed by k object query vectors of a decoder to obtain a feature graph of the filtered feature vectors; sorting the filtered feature vectors based on the feature map, and taking the object query vector and the first k feature vectors after sorting as the input of a decoder; and inputting the output of the decoder into a detection head to obtain the human eye closing condition in the frames, and determining whether fatigue driving exists or not based on the number of eye closing frames detected in unit time.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a fatigue driving detection method and system. Background Art

[0002] According to a report by the World Health Organization (WHO), fatigue driving causes a large number of deaths and injuries every year, especially when driving for a long time, the driver's attention is easily reduced, the reaction speed is significantly slowed down, and the judgment and decision-making ability are seriously affected. Fatigue driving not only causes the driver to deviate from the judgment of road conditions in a short period of time, but also directly increases the probability of traffic accidents. Compared with other traffic violations such as drunk driving and speeding, fatigue driving is more hidden, and the driver may continue driving without realizing that he is tired, which makes it more dangerous.

[0003] Fatigue driving monitoring is mainly divided into local and remote monitoring. Local monitoring uses the vehicle system to identify fatigue driving, while remote monitoring transmits the video to a remote server for monitoring. Local monitoring can significantly reduce the time overhead of data transmission, ensuring that the system can complete fatigue status identification in a short time, thereby reducing the warning opportunities that may be missed due to data delays. In the vehicle environment, computing resources are relatively limited. How to improve the detection speed while ensuring accuracy and limited vehicle resources is crucial. Summary of the invention

[0004] In order to improve the monitoring speed, a first aspect of the present invention provides a fatigue driving detection method, the method comprising: Obtain the distance range of the center of the human eye frame marked in the training sample in the video frame, divide each frame of the driver video shot into multiple image blocks, fuse the image blocks and the position encoding of the image blocks, input them into the encoder to obtain the feature vector output by the encoder, cluster the feature vectors, and filter the feature vectors according to the clustering results and the positions of the image blocks corresponding to the feature vectors in the frame and the distance range; The feature matrix composed of the filtered feature vectors is multiplied by the transpose of the matrix composed of the k object query vectors of the decoder to obtain a feature map of the filtered feature vectors; the filtered feature vectors are sorted based on the feature map, and the object query vector and the first k feature vectors after sorting are used as the input of the decoder; The output of the decoder is input into the detection head to obtain the eye closure status in the frame, and whether the driver is fatigued is determined based on the number of eye closure frames detected per unit time.

[0005] Preferably, filtering the feature vector according to the clustering result and the position of the image block corresponding to the feature vector in the frame and the distance range is specifically as follows: Calculate the center of the image block corresponding to the feature vector, and calculate the center of each cluster in the clustering result; Adjusting the similarity between the feature vectors in the cluster according to the center distance of the image blocks corresponding to the two feature vectors and the distance range; The average value of the adjusted similarities between all feature vectors in the cluster is calculated, and the feature vectors far from the cluster center are filtered out from the two feature vectors whose adjusted similarities are greater than the average value.

[0006] Preferably, the similarity between the feature vectors in the cluster is adjusted according to the center distance of the image blocks corresponding to the two feature vectors and the distance range, specifically: If the center distance of the image blocks corresponding to the two eigenvectors is within the distance range, the adjustment factor is set to 0; If the center distance of the image blocks corresponding to the two feature vectors is not within the distance range, the farthest distance between the centers of the two image blocks in the video frame is obtained, the similarity between the two feature vectors in the cluster is calculated, and the ratio of the distance between the centers of the image blocks corresponding to the two feature vectors and the farthest distance is calculated; 1 minus the ratio is used as an adjustment factor for the similarity; The product of the similarity and the adjustment factor is taken as the similarity adjustment result.

[0007] Preferably, the filtered feature vectors are sorted based on the feature graph, specifically: Calculate the average value of each row in the feature map after removing the maximum value, and calculate the difference between the maximum value and the average value; Sort the rows of the feature map in descending order of the difference values; or, The data in each row of the feature map is divided into two categories according to size, the absolute difference of the mean value of each category and the difference in the number of data in each category are calculated, and the rows of the feature map are sorted in descending order according to the product of the absolute difference and the difference.

[0008] Preferably, the determining whether driving is fatigued based on the number of closed-eye frames detected per unit time is specifically as follows: If the number of closed-eye frames detected per unit time is greater than the threshold, a fatigue driving prompt will be given, otherwise no prompt will be given.

[0009] In a second aspect of the present invention, a fatigue driving detection system is provided, the system comprising: A filtering module is used to obtain the distance range of the center of the human eye frame marked in the training sample in the video frame, divide each frame of the driver video shot into multiple image blocks, fuse the image blocks and the position codes of the image blocks, input them into the encoder to obtain the feature vector output by the encoder, cluster the feature vector, and filter the feature vector according to the clustering result and the position of the image block corresponding to the feature vector in the frame and the distance range; A decoding module is used to obtain a feature map of the filtered feature vector by multiplying the feature matrix composed of the filtered feature vectors and the transpose of the matrix composed of the k object query vectors of the decoder; sorting the filtered feature vectors based on the feature map, and taking the object query vector and the first k feature vectors after sorting as the input of the decoder; The detection module is used to input the output of the decoder into the detection head to obtain the eye closure status in the frame, and determine whether the driver is fatigued based on the number of eye closure frames detected per unit time.

[0010] Preferably, filtering the feature vector according to the clustering result and the position of the image block corresponding to the feature vector in the frame and the distance range is specifically as follows: Calculate the center of the image block corresponding to the feature vector, and calculate the center of each cluster in the clustering result; Adjusting the similarity between the feature vectors in the cluster according to the center distance of the image blocks corresponding to the two feature vectors and the distance range; The average value of the adjusted similarities between all feature vectors in the cluster is calculated, and the feature vectors far from the cluster center are filtered out from the two feature vectors whose adjusted similarities are greater than the average value.

[0011] Preferably, the similarity between the feature vectors in the cluster is adjusted according to the center distance of the image blocks corresponding to the two feature vectors and the distance range, specifically: If the center distance of the image blocks corresponding to the two eigenvectors is within the distance range, the adjustment factor is set to 0; If the center distance of the image blocks corresponding to the two feature vectors is not within the distance range, the farthest distance between the centers of the two image blocks in the video frame is obtained, the similarity between the two feature vectors in the cluster is calculated, and the ratio of the distance between the centers of the image blocks corresponding to the two feature vectors and the farthest distance is calculated; 1 minus the ratio is used as an adjustment factor for the similarity; The product of the similarity and the adjustment factor is taken as the similarity adjustment result.

[0012] Preferably, the filtered feature vectors are sorted based on the feature graph, specifically: Calculate the average value of each row in the feature map after removing the maximum value, and calculate the difference between the maximum value and the average value; Sort the rows of the feature map in descending order of the difference values; or, The data in each row of the feature map is divided into two categories according to size, the absolute difference of the mean value of each category and the difference in the number of data in each category are calculated, and the rows of the feature map are sorted in descending order according to the product of the absolute difference and the difference.

[0013] Preferably, the determining whether driving is fatigued based on the number of closed-eye frames detected per unit time is specifically as follows: If the number of closed-eye frames detected per unit time is greater than the threshold, a fatigue driving prompt will be given, otherwise no prompt will be given.

[0014] A third aspect of the present invention provides a computer program, which, when executed by a processor, implements the method according to the first aspect of the present invention.

[0015] In order to reduce the computational complexity of the decoder in fatigue driving monitoring, the present invention filters the feature vectors output by the encoder according to clustering and the distance between the feature vectors and the image blocks corresponding to the feature vectors in the video frame, and filters out the feature vectors that are very close and very similar; then, the feature map of the feature vector is obtained according to the relationship between the feature vector and the object query vector, and k object query vectors and k feature vectors selected using the feature map are used as the input of the decoder. Through the above process, the number of feature vectors input to the decoder is reduced, the computational complexity of the decoder is reduced, and the monitoring speed is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flow chart of Embodiment 1; Figure 2 is a schematic diagram of the encoder; Figure 3 is a schematic diagram of a decoder; Figure 4 This is a schematic diagram of the recognition results. DETAILED DESCRIPTION

[0017] In order to enable persons skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only one or more partial embodiments of this specification, rather than all embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.

[0018] It should be noted that, in the absence of conflict, one or more embodiments and features in the embodiments in this specification can be combined with each other. One or more embodiments of this specification will be described in detail below with reference to the drawings and in combination with the embodiments.

[0019] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0020] Embodiment 1: This embodiment provides a method for detecting fatigue driving. Figure 1 As shown, the method includes: S1, obtaining the distance range of the center of the human eye frame marked in the training sample in the video frame, dividing each frame of the driver video shot into multiple image blocks, fusing the image blocks and the position encoding of the image blocks, and inputting them into the encoder to obtain the feature vector output by the encoder, clustering the feature vector, and filtering the feature vector according to the clustering result and the position of the image block corresponding to the feature vector in the frame and the distance range; The human eyes are annotated in the training samples, and the open eyes, closed eyes, etc. are marked. Since the distance between the two eyes of a person is generally fixed and the difference between people is not big, especially in video images, the distance between the two eyes of a person in the video frame can be obtained from the center of the annotated human eye frame in the training samples. The distance of the human eye sockets here refers to the distance between the centers of the annotated frames of the two eyes of a person in the same frame.

[0021] Each frame of the video is divided into multiple small blocks, which are image blocks. For each image block, after the image block is flattened, its position code in the frame is added, and then these image blocks containing position information are sent to the encoder. The encoder preferably uses the encoder of the DETR model, and the encoder of the ViT model can also be used. The encoder will output multiple feature vectors. In one embodiment, the number and dimension of the input features and output features of the encoder are the same. Each input image block corresponds to a feature vector output by the encoder, such as Figure 2 shown.

[0022] After obtaining the feature vectors, the feature vectors are clustered, and the clustering methods include but are not limited to K-means clustering, hierarchical clustering, DBSCAN clustering, etc. After clustering, similar feature vectors will be clustered into one category. Feature vectors that are close and similar often represent an object in decoding. In order to reduce the amount of decoded data, the feature vectors are filtered, and the feature vectors that belong to the same cluster and have a similarity greater than a preset value, and the position of the database corresponding to the feature vector in the frame is less than a fixed value are filtered.

[0023] In another embodiment, filtering the feature vector according to the clustering result and the position of the image block corresponding to the feature vector in the frame and the distance range is specifically: Calculate the center of the image block corresponding to the feature vector, and calculate the center of each cluster in the clustering result; Adjusting the similarity between the feature vectors in the cluster according to the center distance of the image blocks corresponding to the two feature vectors and the distance range; The average value of the adjusted similarities between all feature vectors in the cluster is calculated, and the feature vectors far from the cluster center are filtered out from the two feature vectors whose adjusted similarities are greater than the average value.

[0024] For each feature vector, calculate the center position of its corresponding image block. The center position of the image block can be represented by the coordinates of the image block in the video frame. Calculate the center position of each cluster. For each pair of feature vectors in the same cluster, calculate the distance between the centers of the corresponding image blocks and calculate the similarity of the pair of feature vectors. Adjust the similarity between the feature vectors in the cluster according to the center distance of the image blocks corresponding to the two feature vectors and the distance range, and further calculate the average value of the adjusted similarity of all feature vector pairs in the cluster. If the adjusted similarity is greater than the average value, filter out the feature vector farthest from the cluster center in the pair of feature vectors. In this way, some feature vectors that are very close and very similar can be filtered out, reducing the amount of data calculated by the decoder.

[0025] In yet another embodiment, the similarity between the feature vectors in the cluster is adjusted according to the center distance of the image blocks corresponding to the two feature vectors and the distance range, specifically: If the center distance of the image blocks corresponding to the two eigenvectors is within the distance range, the adjustment factor is set to 0; If the center distance of the image blocks corresponding to the two feature vectors is not within the distance range, the farthest distance between the centers of the two image blocks in the video frame is obtained, the similarity between the two feature vectors in the cluster is calculated, and the ratio of the distance between the centers of the image blocks corresponding to the two feature vectors and the farthest distance is calculated; 1 minus the ratio is used as an adjustment factor for the similarity; The product of the similarity and the adjustment factor is taken as the similarity adjustment result.

[0026] For any two feature vectors in the cluster, the distance between the center positions of their corresponding image blocks is first calculated. If the distance is within the preset distance range, the adjustment factor is set to 0, indicating that no similarity adjustment is required between the two feature vectors. The distance range can avoid filtering the feature vectors of the two eye parts.

[0027] If the distance between the image block centers corresponding to the two eigenvectors is not within the preset distance range, the farthest distance between the centers of all image blocks in the video frame is obtained, and the ratio of the distance to the farthest distance is calculated according to the actual distance between the center positions of the image blocks corresponding to the two eigenvectors. The similarity adjustment factor is subtracted from 1 to obtain the similarity value of the eigenvectors with a long distance between the image blocks.

[0028] S2, multiplying the feature matrix composed of the filtered feature vectors by the transpose of the matrix composed of the k object query vectors of the decoder to obtain a feature map of the filtered feature vectors; sorting the filtered feature vectors based on the feature map, and taking the object query vector and the first k feature vectors after sorting as the input of the decoder; After filtering, for the retained feature vectors, the feature vectors are concatenated row by row to obtain a feature matrix, and the feature matrix is ​​calculated and multiplied by the transpose of the matrix composed of k object query vectors of the decoder to obtain a feature map of the feature vector. Among them, the dimension of the feature vector is the same as the dimension of the object query vector. After the k object query vectors are concatenated row by row, they are transposed and multiplied with the feature matrix. For example, the feature matrix is ​​d×5, that is, there are d filtered feature vectors; the matrix composed of k object query vectors is k×5, which is 5×k after transposition. The result of the transpose multiplication of the feature matrix and the matrix composed of k object query vectors is d×k, that is, the size of the feature map is d×k, where each row in the feature map is a vector, and the elements in the vector of each row represent the similarity or correlation between the feature matrix and the object query vector. For example, the third element of the second row of the feature map represents the similarity or correlation between the second feature vector after filtering and the third object query vector. The principle here is the same as the principle of calculating the attention score based on the query matrix Q and the key matrix in the attention mechanism. In order to prevent overfitting, in one embodiment, the feature matrix composed of the filtered feature vectors and the transpose of the matrix composed of the k object query vectors of the decoder are multiplied to obtain the feature map of the filtered feature vectors, the feature matrix composed of the filtered feature vectors is multiplied by a trainable weight matrix, and the matrix composed of the k object query vectors of the decoder is also multiplied by another trainable weight matrix, and then the feature matrix multiplied by the trainable weight matrix and the transpose of the matrix composed of the k object query vectors multiplied by the trainable weight matrix are multiplied. Among them, the object query vector is also called Object Query. When there are multiple object query vectors, they are called Object Queries. Figure 3 As shown. Unless otherwise specified, the object query vector in the present invention and the Object Query in the DETR model have the same meaning and the same function. In DETR, each Object Query can be regarded as a representation of a target, which performs a similarity comparison with each position of the prediction result to find the best matching prediction result. Wherein, k is a positive integer greater than 2.

[0029] If a feature vector and an object query vector identify an identification box in the subsequent determination of the eye state, the similarity between this feature vector and other object query matrices will generally be different. The relationship between the feature vector and different object query vectors can be seen through the feature map. Based on this, the filtered feature vector is further filtered based on the feature map. Specifically, in one embodiment, the average value of each row in the feature map after removing the maximum value is calculated, and the difference between the maximum value and the average value is calculated; the rows of the feature map are sorted in descending order according to the difference.

[0030] For each row in the feature map, calculate the average value of this row after removing the maximum value, and calculate the difference between the maximum value and the average value of this row. The larger the difference, the more obvious the difference between this feature vector and different object query vectors. If one feature value is large and the other feature values ​​are small, this feature vector is likely to be the feature vector corresponding to the target frame, that is, the eye frame. On the contrary, if it is relatively uniform, then this feature vector is similar for all object query vectors, and this feature vector is likely to have no corresponding target frame. Then, sort the rows of the feature map in descending order of difference. For example, if the difference of the first row in the feature map is 1 and the difference of the second row is 4, then the second row is ranked before the first row.

[0031] In another embodiment, the data in each row of the feature graph is divided into two categories according to size, the absolute difference of the mean value of each category and the difference in the number of data in each category are calculated, and the rows of the feature graph are sorted in order from large to small according to the product of the absolute difference and the difference.

[0032] Since there are multiple target query vectors, there may be more than one large value in the feature map. Based on this, the data in each row is divided into two categories according to size, such as feature Figure 1 If a row has 5 elements, 1.2, 1, 4, 1.9, and 4.3, 4 and 4.3 will be classified into one category, and 1.2, 1, and 1.9 will be classified into another category. Calculate the absolute difference of the mean value of each category. According to the above example, the absolute difference is 2.78, the number difference is 1, and the final product is 2.78. The product of other rows of the feature map can be calculated in the same way, and then the rows of the feature map can be sorted according to the product.

[0033] Select the first k feature vectors from the sorted rows. These k feature vectors will be input into the decoder together with the k object query vectors, such as Figure 3 The invention does not limit the specific structure of the decoder, and preferably adopts the decoder in the DETR model, or the Transformer Decoder.

[0034] S3, input the output of the decoder to the detection head to obtain the eye closure status in the frame, and determine whether the driver is fatigued based on the number of eye closure frames detected per unit time.

[0035] The vector output by the decoder is passed through FFN to obtain the human eye state corresponding to each output vector of the decoder. Figure 4 The monitoring results of eyes open and eyes closed are shown. Whether driving fatigue is detected is determined based on the PERCLOS fatigue detection criteria. Specifically, if the number of closed-eye frames detected per unit time is greater than a threshold, fatigue driving is prompted, otherwise no prompt is given.

[0036] Embodiment 2: This embodiment provides a fatigue driving detection system, the system comprising: A filtering module is used to obtain the distance range of the center of the human eye frame marked in the training sample in the video frame, divide each frame of the driver video shot into multiple image blocks, fuse the image blocks and the position codes of the image blocks, input them into the encoder to obtain the feature vector output by the encoder, cluster the feature vector, and filter the feature vector according to the clustering result and the position of the image block corresponding to the feature vector in the frame and the distance range; A decoding module is used to obtain a feature map of the filtered feature vector by multiplying the feature matrix composed of the filtered feature vectors and the transpose of the matrix composed of the k object query vectors of the decoder; sorting the filtered feature vectors based on the feature map, and taking the object query vector and the first k feature vectors after sorting as the input of the decoder; The detection module is used to input the output of the decoder into the detection head to obtain the eye closure status in the frame, and determine whether the driver is fatigued based on the number of eye closure frames detected per unit time.

[0037] Preferably, filtering the feature vector according to the clustering result and the position of the image block corresponding to the feature vector in the frame and the distance range is specifically as follows: Calculate the center of the image block corresponding to the feature vector, and calculate the center of each cluster in the clustering result; Adjusting the similarity between the feature vectors in the cluster according to the center distance of the image blocks corresponding to the two feature vectors and the distance range; The average value of the adjusted similarities between all feature vectors in the cluster is calculated, and the feature vectors far from the cluster center are filtered out from the two feature vectors whose adjusted similarities are greater than the average value.

[0038] Preferably, the similarity between the feature vectors in the cluster is adjusted according to the center distance of the image blocks corresponding to the two feature vectors and the distance range, specifically: If the center distance of the image blocks corresponding to the two eigenvectors is within the distance range, the adjustment factor is set to 0; If the center distance of the image blocks corresponding to the two feature vectors is not within the distance range, the farthest distance between the centers of the two image blocks in the video frame is obtained, the similarity between the two feature vectors in the cluster is calculated, and the ratio of the distance between the centers of the image blocks corresponding to the two feature vectors and the farthest distance is calculated; 1 minus the ratio is used as an adjustment factor for the similarity; The product of the similarity and the adjustment factor is taken as the similarity adjustment result.

[0039] Preferably, the filtered feature vectors are sorted based on the feature graph, specifically: Calculate the average value of each row in the feature map after removing the maximum value, and calculate the difference between the maximum value and the average value; Sort the rows of the feature map in descending order of the difference values; or, The data in each row of the feature map is divided into two categories according to size, the absolute difference of the mean value of each category and the difference in the number of data in each category are calculated, and the rows of the feature map are sorted in descending order according to the product of the absolute difference and the difference.

[0040] Preferably, the determining whether driving is fatigued based on the number of closed-eye frames detected per unit time is specifically as follows: If the number of closed-eye frames detected per unit time is greater than the threshold, a fatigue driving prompt will be given, otherwise no prompt will be given.

[0041] Embodiment 3: This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in Embodiment 1 is implemented.

[0042] Embodiment 4: This embodiment provides a computer program, which implements the method described in Embodiment 1 when executed by a processor.

[0043] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, playback end, image acquisition end, executable program code, and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0044] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0045] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for detecting fatigue driving, characterized in that: The method comprises: Obtain the distance range of the center of the human eye frame marked in the training sample in the video frame, divide each frame of the driver video shot into multiple image blocks, fuse the image blocks and the position encoding of the image blocks, input them into the encoder to obtain the feature vector output by the encoder, cluster the feature vectors, and filter the feature vectors according to the feature vector clustering results and the positions of the image blocks corresponding to the feature vectors in the frame and the distance range; The feature matrix composed of the filtered feature vectors is multiplied by the transpose of the matrix composed of the k object query vectors of the decoder to obtain a feature map of the filtered feature vectors; the filtered feature vectors are sorted based on the feature map, and the object query vector and the first k feature vectors after sorting are used as the input of the decoder; The output of the decoder is input into the detection head to obtain the eye closure status in the frame, and whether the driver is fatigued is determined based on the number of eye closure frames detected per unit time.

2. The method according to claim 1, characterized in that The filtering of the feature vector according to the clustering result and the position of the image block corresponding to the feature vector in the frame and the distance range is specifically as follows: Calculate the center of the image block corresponding to the feature vector, and calculate the center of each cluster in the clustering result; Adjusting the similarity between the feature vectors in the cluster according to the center distance of the image blocks corresponding to the two feature vectors and the distance range; The average value of the adjusted similarities between all feature vectors in the cluster is calculated, and the feature vectors far from the cluster center are filtered out from the two feature vectors whose adjusted similarities are greater than the average value.

3. The method according to claim 2, characterized in that The similarity between the feature vectors in the cluster is adjusted according to the center distance of the image blocks corresponding to the two feature vectors and the distance range, specifically: If the center distance of the image blocks corresponding to the two eigenvectors is within the distance range, the adjustment factor is set to 0; If the center distance of the image blocks corresponding to the two feature vectors is not within the distance range, the farthest distance between the centers of the two image blocks in the video frame is obtained, the similarity between the two feature vectors in the cluster is calculated, and the ratio of the distance between the centers of the image blocks corresponding to the two feature vectors and the farthest distance is calculated; 1 minus the ratio is used as an adjustment factor for the similarity; The product of the similarity and the adjustment factor is taken as the similarity adjustment result.

4. The method according to claim 1, characterized in that The filtered feature vectors are sorted based on the feature graph, specifically: Calculate the average value of each row in the feature map after removing the maximum value, and calculate the difference between the maximum value and the average value; Sort the rows of the feature map in descending order of the difference values; or, The data in each row of the feature map is divided into two categories according to size, the absolute difference of the mean value of each category and the difference in the number of data in each category are calculated, and the rows of the feature map are sorted in descending order according to the product of the absolute difference and the difference.

5. The method according to claim 1, characterized in that The determination of fatigue driving based on the number of closed-eye frames detected within a unit time is specifically as follows: If the number of closed-eye frames detected per unit time is greater than the threshold, a fatigue driving prompt will be given, otherwise no prompt will be given.

6. A fatigue driving detection system, characterized in that: The system comprises: A filtering module is used to obtain the distance range of the center of the human eye frame marked in the training sample in the video frame, divide each frame of the driver video shot into multiple image blocks, fuse the image blocks and the position codes of the image blocks, input them into the encoder to obtain the feature vector output by the encoder, cluster the feature vector, and filter the feature vector according to the clustering result and the position of the image block corresponding to the feature vector in the frame and the distance range; A decoding module is used to obtain a feature map of the filtered feature vector by multiplying the feature matrix composed of the filtered feature vectors and the transpose of the matrix composed of the k object query vectors of the decoder; sorting the filtered feature vectors based on the feature map, and taking the object query vector and the first k feature vectors after sorting as the input of the decoder; The detection module is used to input the output of the decoder into the detection head to obtain the eye closure status in the frame, and determine whether the driver is fatigued based on the number of eye closure frames detected per unit time.

7. The system according to claim 6, characterized in that The filtering of the feature vector according to the clustering result and the position of the image block corresponding to the feature vector in the frame and the distance range is specifically as follows: Calculate the center of the image block corresponding to the feature vector, and calculate the center of each cluster in the clustering result; Adjusting the similarity between the feature vectors in the cluster according to the center distance of the image blocks corresponding to the two feature vectors and the distance range; The average value of the adjusted similarities between all feature vectors in the cluster is calculated, and the feature vectors far from the cluster center are filtered out from the two feature vectors whose adjusted similarities are greater than the average value.

8. The system according to claim 7, characterized in that The similarity between the feature vectors in the cluster is adjusted according to the center distance of the image blocks corresponding to the two feature vectors and the distance range, specifically: If the center distance of the image blocks corresponding to the two eigenvectors is within the distance range, the adjustment factor is set to 0; If the center distance of the image blocks corresponding to the two feature vectors is not within the distance range, the farthest distance between the centers of the two image blocks in the video frame is obtained, the similarity between the two feature vectors in the cluster is calculated, and the ratio of the distance between the centers of the image blocks corresponding to the two feature vectors and the farthest distance is calculated; 1 minus the ratio is used as an adjustment factor for the similarity; The product of the similarity and the adjustment factor is taken as the similarity adjustment result.

9. The system according to claim 6, characterized in that The filtered feature vectors are sorted based on the feature graph, specifically: Calculate the average value of each row in the feature map after removing the maximum value, and calculate the difference between the maximum value and the average value; Sort the rows of the feature map in descending order of the difference values; or, The data in each row of the feature map is divided into two categories according to size, the absolute difference of the mean value of each category and the difference in the number of data in each category are calculated, and the rows of the feature map are sorted in descending order according to the product of the absolute difference and the difference.

10. A computer program, characterized in that When the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 5.

Images