Fatigue driving detection method and fatigue noise addition model construction method

Abstract

This application proposes a method and device for constructing a fatigue driving detection model based on an adaptive gating strategy, including the following steps: obtaining a driving video of a vehicle to be detected in real time, inputting the driving video of the vehicle to be detected into a pre-trained fatigue noise adding model to add fatigue noise to obtain a noisy driving video; analyzing the feature difference between the noisy driving video and the driving video of the vehicle to be detected, and if the feature difference between the noisy driving video and the driving video of the vehicle to be detected is greater than a set threshold, it is determined that the vehicle in the driving video of the vehicle to be detected is fatigued. This solution uses an adaptive gating strategy to increase the weight of important features and suppress features that are not related to fatigue driving to achieve the effect of improving detection accuracy.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to a method for detecting fatigue driving and a method for constructing a fatigue noise addition model. Background Art

[0002] On highways, fatigue driving is one of the main causes of traffic accidents, accounting for about 20%-30% of traffic accidents, and even more than 30% in certain circumstances. When the driver is in a fatigued state, judgment, reaction and operating ability are affected. Mild fatigue can lead to untimely and inaccurate gear shifting, moderate fatigue can make the operator sluggish or even forget to operate, severe fatigue can cause subconscious operation, short-term sleep, and loss of control of the vehicle in severe cases. At the same time, fatigue is also accompanied by blurred vision, back pain, lack of concentration and other phenomena. At this time, if you force yourself to drive, it is very likely to cause an accident.

[0003] The existing technology generally uses a camera facing the cab to perform visual detection of the operator's face when detecting fatigue driving, such as judging the increase in blinking frequency, the increase in eye closure ratio, the increase in yawning frequency and abnormal head posture to determine whether the operator is driving fatigued, thereby issuing an alarm. However, not every vehicle is equipped with a fatigue detection camera in the cab. In the case that fatigue driving cameras are not popular, fatigue driving is generally detected through the vehicle's driving information, such as judging whether the driver is fatigued based on information such as the driving route, driving time or driving speed. However, this method may lead to misjudgment due to situations such as changing drivers, thereby greatly reducing the detection results.

[0004] In summary, there is an urgent need for a method that can accurately judge driver fatigue driving through lane line information and vehicle driving trajectory. Summary of the invention

[0005] The embodiments of the present application provide a method for detecting fatigue driving and a method for constructing a fatigue noise addition model. A camera is used to detect abnormal driving behavior of a vehicle to determine fatigue driving, and an adaptive gating strategy is used to increase the weights of important features and suppress features that are not related to fatigue driving to achieve the effect of improving detection accuracy.

[0006] In a first aspect, an embodiment of the present application provides a method for detecting fatigue driving, the method comprising:

[0007] Acquire a driving video of a to-be-detected vehicle in real time, input the driving video of the to-be-detected vehicle into a pre-trained fatigue noise adding model, add fatigue noise to obtain a noisy driving video, wherein the fatigue noise is a vehicle driving feature simulating fatigue driving, wherein the fatigue noise adding model adds fatigue noise in a stepwise constrained reconstruction manner, and in the stepwise constrained reconstruction process, adjusts the importance of different features in each time step based on an adaptive gating mechanism;

[0008] The feature difference between the noisy driving video and the driving video of the vehicle to be detected is analyzed. If the feature difference between the noisy driving video and the driving video of the vehicle to be detected is greater than a set threshold, it is determined that the vehicle in the driving video of the vehicle to be detected is fatigued.

[0009] In a second aspect, an embodiment of the present application provides a method for constructing a fatigue noise addition model, comprising:

[0010] Constructing a fatigue noise adding framework, wherein the fatigue noise adding framework includes a feature extraction module, a noise adding module and a coding and decoding module;

[0011] Acquire multiple training samples, where the training samples are normal driving videos of vehicles acquired by a surveillance camera, and the feature extraction module extracts features from each training sample to obtain a first video feature corresponding to each training sample;

[0012] Constructing fatigue noise, and adding the fatigue noise to the first video feature in the noise adding module to obtain a second video feature, wherein the fatigue noise is a vehicle driving feature simulating fatigue driving;

[0013] In the coding model, based on the context embedding of the second video feature at the current time step, the second video feature is subjected to stepwise constrained reconstruction to obtain a third video feature, wherein an adaptive gating mechanism is used to adjust the importance of different features in each time step during the stepwise constrained reconstruction process;

[0014] A loss function is used to measure the difference between the third video feature and the corresponding training sample, and the parameters of the fatigue noise adding framework are adjusted based on the size of the difference until the set conditions are met to complete the training and obtain the fatigue noise adding model.

[0015] In a third aspect, an embodiment of the present application provides a fatigue driving detection device, comprising:

[0016] an acquisition module, for acquiring a driving video of a to-be-detected vehicle in real time, and inputting the driving video of the to-be-detected vehicle into a pre-trained fatigue noise adding model to add fatigue noise to obtain a noisy driving video, wherein the fatigue noise is a driving feature of a vehicle simulating fatigue driving, wherein the fatigue noise adding model adds fatigue noise in a stepwise constrained reconstruction manner, and in the stepwise constrained reconstruction process, adjusts the importance of different features in each time step based on an adaptive gating mechanism;

[0017] The fatigue detection module is used to analyze the feature difference between the noisy driving video and the driving video of the vehicle to be detected. If the feature difference between the noisy driving video and the driving video of the vehicle to be detected is greater than a set threshold, it is determined that the vehicle in the driving video of the vehicle to be detected is driving fatigued.

[0018] In a fourth aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute a fatigue driving detection method or a fatigue noise addition model construction method.

[0019] In a fifth aspect, an embodiment of the present application provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process, and the process includes a method for constructing a fatigue driving detection method or a fatigue noise addition model.

[0020] The main contributions and innovations of the present invention are as follows:

[0021] The embodiment of the present application uses multiple feature spaces with different numbers of channels to extract features from training samples, which can integrate rich feature information from low levels to high levels, so that the subsequent network can use these multi-scale features to better understand the content of the image, thereby improving the feature representation capability of high-resolution images to adapt to complex image tasks; the embodiment of the present application adds fatigue noise to allow the fatigue detection model to learn feature changes under different fatigue driving conditions and better cope with abnormal situations in real scenes. It helps the model to identify the driving characteristics of fatigue driving and avoid overfitting of the model to normal driving conditions; the embodiment of the present application extracts the potential features of each time step in the second video feature to complete the compressed abstraction of the second feature image, extracts the key information in each time step, removes redundancy, and enables subsequent processing to focus more on the core features related to fatigue driving, thereby improving processing efficiency.

[0022] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0024] Figure 1 is a flow chart of a fatigue driving detection method according to an embodiment of the present application;

[0025] Figure 2 is a structural block diagram of a fatigue driving detection device according to an embodiment of the present application;

[0026] Figure 3 It is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0028] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0029] Embodiment 1

[0030] The embodiment of the present application provides a fatigue driving detection method, which uses a camera to detect abnormal driving behavior of a vehicle to judge fatigue driving, and uses an adaptive gating strategy to increase the weight of important features and suppress features that are not related to fatigue driving to achieve the effect of improving detection accuracy. Specifically, reference Figure 1 , the method comprising:

[0031] Acquire a driving video of a to-be-detected vehicle in real time, input the driving video of the to-be-detected vehicle into a pre-trained fatigue noise adding model, add fatigue noise to obtain a noisy driving video, wherein the fatigue noise is a vehicle driving feature simulating fatigue driving, wherein the fatigue noise adding model adds fatigue noise in a stepwise constrained reconstruction manner, and in the stepwise constrained reconstruction process, adjusts the importance of different features in each time step based on an adaptive gating mechanism;

[0032] The feature difference between the noisy driving video and the driving video of the vehicle to be detected is analyzed. If the feature difference between the noisy driving video and the driving video of the vehicle to be detected is greater than a set threshold, it is determined that the vehicle in the driving video of the vehicle to be detected is fatigued.

[0033] In addition, the present application also proposes a method for constructing a fatigue noise adding model, by which fatigue noise is added to the characteristics of a normally driving vehicle, so as to detect fatigue driving, the method comprising:

[0034] Constructing a fatigue noise adding framework, wherein the fatigue noise adding framework includes a feature extraction module, a noise adding module and a coding and decoding module;

[0035] Acquire multiple training samples, where the training samples are normal driving videos of vehicles acquired by a surveillance camera, and the feature extraction module extracts features from each training sample to obtain a first video feature corresponding to each training sample;

[0036] Constructing fatigue noise, and adding the fatigue noise to the first video feature in the noise adding module to obtain a second video feature, wherein the fatigue noise is a vehicle driving feature simulating fatigue driving;

[0037] In the coding model, based on the context embedding of the second video feature at the current time step, the second video feature is subjected to stepwise constrained reconstruction to obtain a third video feature, wherein an adaptive gating mechanism is used to adjust the importance of different features in each time step during the stepwise constrained reconstruction process;

[0038] A loss function is used to measure the difference between the third video feature and the corresponding training sample, and the parameters of the fatigue noise adding framework are adjusted based on the size of the difference until the set conditions are met to complete the training and obtain the fatigue noise adding model.

[0039] In some specific embodiments, when obtaining training samples, the present solution needs to collect highway driving video images covering various lighting conditions such as evening, early morning, rainy and snowy weather, and the training samples need to include multiple high-speed scenes such as main lines, interchanges, bridges and tunnels, so as to improve the generalization ability of the model, enhance the robustness of the fatigue driving detection model, and enable the fatigue driving detection model to be applied in multiple scenarios.

[0040] Specifically, the image resolution of the training samples obtained by this solution is 1920*1080.

[0041] In some embodiments, in the feature extraction module, feature extraction is performed on the training sample through a plurality of feature spaces with different numbers of channels connected in series, and then the features extracted in each feature space are spliced ​​to obtain the first video feature.

[0042] In this scheme, three feature spaces are used to extract features from training samples, and the three feature spaces are connected in series in the order of the number of channels from small to large. The number of channels in the first feature space is 64, the number of channels in the second feature space is 128, and the number of channels in the third feature space is 256. The formula for extracting features from training samples by the feature extraction module is as follows:

[0043]

[0044] in, is the first video feature, is the number of channels of the first video feature, H is the height of the first video feature, W is the width of the first video feature, concat means concatenation, represents k feature spaces. Since the image resolution of the training samples obtained in this scheme is 1920*1080, k is 3 in this scheme. is a feature space with 64 channels. The extracted feature map size is 960*540. is a feature space with 128 channels. The extracted feature map size is 480*270. is a feature space with 256 channels. The extracted feature map size is 240*135.

[0045] Specifically, by extracting features from training samples through multiple feature spaces with different numbers of channels, it is possible to integrate rich feature information from low-level to high-level, so that the subsequent network can use these multi-scale features to better understand the content of the image, thereby improving the feature representation ability of high-resolution images to adapt to complex image tasks.

[0046] In some embodiments, a noise matrix is ​​constructed, the fatigue noise is first added to the first video feature to obtain a first noisy video feature, and then the first noisy video feature is converted by the noise matrix to obtain a second video feature.

[0047] Specifically, the formula for adding noise to the first video feature is expressed as follows:

[0048]

[0049] in, is the second video feature, is the number of channels of the second video feature, H is the height of the second video feature, W is the width of the second video feature, is the noise matrix, and is a transformation matrix, Fatigue noise.

[0050] Specifically, by adding fatigue noise, the fatigue detection model can learn the feature changes under different fatigue driving conditions and better deal with abnormal situations in real scenarios. It helps the model identify the driving characteristics of fatigue driving and avoids overfitting of the model to normal driving conditions.

[0051] In some specific embodiments, the judgment of fatigue driving in this solution is generally determined by the following driving behaviors:

[0052] 1. Frequent lane changes: From the time the front wheels of the vehicle touch the lane line to the time the rear wheels of the vehicle completely leave the lane, this process is defined as a lane change behavior. At the same time, the lane change behavior is a process from start to finish, and the duration of the entire process is T. After a large number of video studies and analyses, when T is less than 3 seconds, it is considered a frequent lane change.

[0053] 2. Continuous lane change: When a vehicle passes through two or more different lanes continuously from a certain lane within a duration T.

[0054] 3. S-shaped lane change: When a vehicle changes lanes continuously around a lane line, that is, the vehicle changes from one lane to the adjacent lane, and then changes back to the original lane from the adjacent lane.

[0055] Of course, the above are just examples to illustrate possible fatigue driving behaviors. In actual applications, fatigue noise includes more fatigue driving behaviors, such as speed changes.

[0056] In some embodiments, a multi-layer transformer is used in the encoding and decoding module to encode the second video feature to obtain an encoding result, and then the encoding result is decoded to obtain the potential feature of each time step, and an adaptive gating function corresponding to each time step is generated for the potential feature of each time step based on the context embedding of each time step, and the potential feature of the corresponding time step is adjusted based on the adaptive gating function to obtain the updated feature of each time step, and the updated features of each time step are integrated to obtain the third video feature.

[0057] Specifically, since this solution uses video as input, the time step in this solution represents each frame image in the video, or represents a set of images in each period of time.

[0058] Specifically, this solution uses a three-layer transformer encoder to encode the second video feature to obtain an encoding result, which is expressed as follows:

[0059]

[0060] in, is the encoding result, Represents a three-layer Transformer encoder.

[0061] Furthermore, in the step of "decoding the encoding results to obtain potential features of each time step", if the current time step is the first time step, the encoding result of the current time step is decoded based on the context embedding of the current time step to obtain the potential features; if the current time step is not the first time step, the encoding result of the current time step is decoded based on the potential features of the previous time step and the context embedding of the current time step to obtain the potential features.

[0062] Specifically, the formula for decoding the encoding result is as follows:

[0063]

[0064] in, Indicates that the decoder is at the current time step potential characteristics, Represents the time step The output features of is the context embedding of the current time step t, represents the latent features at the previous time step.

[0065] Specifically, this solution completes the compressed abstraction of the second video features by extracting the potential features of each time step in the second video features, extracts the key information in each time step, removes redundancy, and makes subsequent processing more focused on the core features related to fatigue driving, thereby improving processing efficiency.

[0066] Furthermore, a gating weight matrix is ​​constructed, and the context embedding of each time step is concatenated with the latent features of the corresponding time step to obtain a concatenated result. The concatenated result is mapped to the feature space via the gating weight matrix and output to obtain an adaptive gating function corresponding to each time step. The adaptive gating function is then element-wise multiplied with the latent features of the corresponding time step to obtain the adaptive latent features of each time step. Transformer is then used to restore the adaptive latent features of each time step to obtain restored features, and the adaptive latent features and restored features of each time step are fused to obtain updated features.

[0067] Specifically, the formulas for obtaining adaptive potential features and restoring features are as follows:

[0068]

[0069]

[0070]

[0071] in, is the adaptive gating function, is the gating weight matrix, For potential characteristics, Contextual embeddings, is the feature fusion operation, is element-wise multiplication, is the adaptive latent feature, To restore the feature.

[0072] Specifically, each adaptive latent feature is restored using the context embedding corresponding to each adaptive latent feature to obtain a restored feature.

[0073] Specifically, there is also a distinction in importance among the potential features in each time step. Some features are highly correlated with fatigue driving, while some features are less correlated with fatigue driving. This scheme uses an adaptive gating function to increase the weight of important features and suppress features that are not related to fatigue driving to achieve the effect of improving detection accuracy.

[0074] Specifically, the formula for obtaining the updated features is as follows:

[0075]

[0076] in, To update the features, is the weight matrix used to adjust Contribution To restore the feature.

[0077] In some specific embodiments, the formula for integrating the updated features of each time step to obtain the third video feature is expressed as follows:

[0078]

[0079] in, For the third video feature, Used to map the updated features back to the original feature space.

[0080] Specifically, since the splicing result is mapped into the feature space in the adaptive gating module, after completing a series of operations in the feature space to obtain updated features, the updated features in the feature space are mapped back to the original feature space for output to obtain the third video feature image.

[0081] In some specific embodiments, a mean square error is used as a loss function to calculate the difference between the third video feature and the corresponding training sample, and the formula is expressed as follows:

[0082]

[0083] Among them, L is the mean square error result, ‖∙‖_2 is the L2 norm, which is used to calculate the Euclidean distance between the training sample and the third video feature.

[0084] Specifically, this scheme uses mean square error to judge the similarity between the third video feature and the training sample, with the aim of ensuring that the noise added by the fatigue driving detection model is based on the training sample itself, rather than added randomly. In other words, the added noise is within a controllable range.

[0085] In some specific embodiments, an anomaly detection module is constructed to analyze the anomaly results of the noisy driving video and the driving video of the vehicle to be detected to generate an anomaly map, and the fatigue driving behavior is intuitively displayed through the anomaly map.

[0086] Specifically, since the fatigue driving detection model in this scheme relies on the driving characteristics of the vehicle itself to add fatigue driving noise, if the fatigue driving tendency in the video to be detected is small, then the noise added to the noisy driving video obtained by adding noise to it in this scheme will be small. Therefore, if the fatigue driving tendency in the video to be detected is large or fatigue driving has already occurred, then the noise added to the noisy driving video obtained by adding noise to it in this scheme will be large, resulting in the situation that the difference between the characteristic image of the noisy driving video and the driving video of the vehicle to be detected is greater than the set threshold and is judged as fatigue driving.

[0087] Embodiment 2

[0088] Based on the same idea, refer to Figure 2, the present application also proposes a fatigue driving detection device, comprising:

[0089] an acquisition module, for acquiring a driving video of a to-be-detected vehicle in real time, and inputting the driving video of the to-be-detected vehicle into a pre-trained fatigue noise adding model to add fatigue noise to obtain a noisy driving video, wherein the fatigue noise is a driving feature of a vehicle simulating fatigue driving, wherein the fatigue noise adding model adds fatigue noise in a stepwise constrained reconstruction manner, and in the stepwise constrained reconstruction process, adjusts the importance of different features in each time step based on an adaptive gating mechanism;

[0090] The fatigue detection module is used to analyze the feature difference between the noisy driving video and the driving video of the vehicle to be detected. If the feature difference between the noisy driving video and the driving video of the vehicle to be detected is greater than a set threshold, it is determined that the vehicle in the driving video of the vehicle to be detected is driving fatigued.

[0091] Embodiment 3

[0092] This embodiment also provides an electronic device, referring to Figure 3 , comprises a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above method embodiments.

[0093] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0094] Among them, the memory 404 may include a large capacity memory 404 for data or instructions. For example, but not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In appropriate cases, the memory 404 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 404 may be inside or outside the data processing device. In a specific embodiment, the memory 404 is a non-volatile memory. In a specific embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0095] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0096] The processor 402 implements any one of the fatigue driving detection methods in the above embodiments by reading and executing computer program instructions stored in the memory 404 .

[0097] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0098] The transmission device 406 can be used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0099] The input and output device 408 is used to input or output information. In this embodiment, the input information may be a driving video of a vehicle to be detected, fatigue noise, etc., and the output information may be a fatigue detection result, etc.

[0100] Optionally, in this embodiment, the processor 402 may be configured to perform the following steps through a computer program:

[0101] Acquire a driving video of a to-be-detected vehicle in real time, input the driving video of the to-be-detected vehicle into a pre-trained fatigue noise adding model, add fatigue noise to obtain a noisy driving video, wherein the fatigue noise is a vehicle driving feature simulating fatigue driving, wherein the fatigue noise adding model adds fatigue noise in a stepwise constrained reconstruction manner, and in the stepwise constrained reconstruction process, adjusts the importance of different features in each time step based on an adaptive gating mechanism;

[0102] The feature difference between the noisy driving video and the driving video of the vehicle to be detected is analyzed. If the feature difference between the noisy driving video and the driving video of the vehicle to be detected is greater than a set threshold, it is determined that the vehicle in the driving video of the vehicle to be detected is fatigued.

[0103] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0104] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0105] Embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, at this point, it should be noted that, for example, Figure 3 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or storage blocks implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0106] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0107] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for constructing a fatigue noise adding model, characterized in that: The following steps are involved: Constructing a fatigue noise adding framework, wherein the fatigue noise adding framework includes a feature extraction module, a noise adding module and a coding and decoding module; Acquire multiple training samples, where the training samples are normal driving videos of vehicles acquired by a surveillance camera, and the feature extraction module extracts features from each training sample to obtain a first video feature corresponding to each training sample; Constructing fatigue noise, and adding the fatigue noise to the first video feature in the noise adding module to obtain a second video feature, wherein the fatigue noise is a vehicle driving feature simulating fatigue driving; In the encoding and decoding module, based on the context embedding of the second video feature at the current time step, the second video feature is subjected to stepwise constrained reconstruction to obtain a third video feature, wherein an adaptive gating mechanism is used to adjust the importance of different features in each time step during the stepwise constrained reconstruction process, and the third video feature is a noisy driving video; A loss function is used to measure the difference between the third video feature and the corresponding training sample, and the parameters of the fatigue noise adding framework are adjusted based on the size of the difference until the set conditions are met to complete the training and obtain the fatigue noise adding model.

2. The method for constructing a fatigue noise adding model according to claim 1, characterized in that: In the encoding and decoding module, a multi-layer transformer is used to encode the second video feature to obtain an encoding result, and then the encoding result is decoded to obtain the potential feature of each time step, and an adaptive gating function corresponding to each time step is generated for the potential feature of each time step based on the context embedding of each time step, and the potential feature of the corresponding time step is adjusted based on the adaptive gating function to obtain the updated feature of each time step, and the updated features of each time step are integrated to obtain the third video feature.

3. The method for constructing a fatigue noise adding model according to claim 2, characterized in that: In the step of "decoding the encoding result to obtain potential features of each time step", if the current time step is the first time step, the encoding result of the current time step is decoded based on the context embedding of the current time step to obtain the potential features; if the current time step is not the first time step, the encoding result of the current time step is decoded based on the potential features of the previous time step and the context embedding of the current time step to obtain the potential features.

4. The method for constructing a fatigue noise adding model according to claim 2, characterized in that: A gating weight matrix is ​​constructed, and the context embedding of each time step is concatenated with the latent features of the corresponding time step to obtain a concatenated result. The concatenated result is mapped to the feature space via the gating weight matrix and output to obtain an adaptive gating function corresponding to each time step. The adaptive gating function is then element-wise multiplied with the latent features of the corresponding time step to obtain the adaptive latent features of each time step. Transformer is then used to restore the adaptive latent features of each time step to obtain restored features, and the adaptive latent features and restored features of each time step are fused to obtain updated features.

5. The method for constructing a fatigue noise adding model according to claim 1, characterized in that: In the feature extraction module, features of training samples are extracted through a plurality of feature spaces with different numbers of channels connected in series, and then the features extracted in each feature space are spliced ​​to obtain the first video feature.

6. The method for constructing a fatigue noise adding model according to claim 1, characterized in that: A noise matrix is ​​constructed, the fatigue noise is first added to the first video feature to obtain a first noisy video feature, and then the first noisy video feature is converted by the noise matrix to obtain a second video feature.

7. A method for detecting fatigue driving, characterized in that: The following steps are involved: Acquire a driving video of a vehicle to be detected in real time, input the driving video of the vehicle to be detected into a fatigue noise adding model constructed by a method described in any one of claims 1 to 6 to obtain a noisy driving video, wherein the fatigue noise is a vehicle driving feature simulating fatigue driving, wherein the fatigue noise adding model adds fatigue noise in a stepwise constrained reconstruction manner, and in the stepwise constrained reconstruction process, adjusts the importance of different features in each time step based on an adaptive gating mechanism; The feature difference between the noisy driving video and the driving video of the vehicle to be detected is analyzed. If the feature difference between the noisy driving video and the driving video of the vehicle to be detected is greater than a set threshold, it is determined that the vehicle in the driving video of the vehicle to be detected is fatigued.

8. A fatigue driving detection device, characterized in that: include: An acquisition module is used to acquire a driving video of a vehicle to be detected in real time, and input the driving video of the vehicle to be detected into a fatigue noise adding model constructed by a method described in any one of claims 1 to 6 to obtain a noisy driving video, wherein the fatigue noise is a vehicle driving feature simulating fatigue driving, wherein the fatigue noise adding model adds fatigue noise in a stepwise constrained reconstruction manner, and in the stepwise constrained reconstruction process, the importance of different features in each time step is adjusted based on an adaptive gating mechanism; The fatigue detection module is used to analyze the feature difference between the noisy driving video and the driving video of the vehicle to be detected. If the feature difference between the noisy driving video and the driving video of the vehicle to be detected is greater than a set threshold, it is determined that the vehicle in the driving video of the vehicle to be detected is driving fatigued.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the method for constructing a fatigue noise addition model described in any one of claims 1 to 6 or the method for detecting fatigue driving described in claim 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute a process, wherein the process includes a method for constructing a fatigue noise addition model according to any one of claims 1 to 6 or a fatigue driving detection method according to claim 7.

Images