Alcohol concentration rapid detection stick monitoring and early warning system based on face detection function

Abstract

The present invention belongs to the field of drunk driving monitoring, and specifically refers to an alcohol concentration rapid detection stick monitoring and early warning system based on face detection function, the system includes: an alcohol concentration rapid detection stick terminal and a cloud server; the alcohol concentration rapid detection stick terminal includes a camera acquisition module, an alcohol detection module, a communication module and a display module; the cloud server includes a fatigue monitoring module and a face matching module. This solution combines the alcohol concentration rapid detection stick with the face detection function to monitor whether the driver is fatigued while detecting drunk driving; introduces the time dimension, makes full use of the video dynamic information contained in continuous image frames, and uses a multi-level adaptive 3D-CNN model to learn the driver's status to improve the performance of identifying fatigue driving; uses a three-stage model with non-intersecting joint suppression technology to improve the flexibility of detection and drive overall efficiency.

Description

Technical Field

[0001] The invention belongs to the field of drunk driving monitoring, and in particular to a monitoring and early warning system for alcohol detection. Background Art

[0002] With the acceleration of urbanization and the increase in the number of vehicles, traffic accidents caused by drunk driving and fatigue driving occur frequently, resulting in serious personal injury and property loss. However, the traditional alcohol concentration rapid detection stick has a single application direction and cannot upload alcohol test results to the cloud in real time to achieve data integration; the traditional fatigue driving method has high computational complexity and low evaluation accuracy; the traditional target detection method has poor readability and cannot perform face recognition in a hierarchical and step-by-step manner. Summary of the invention

[0003] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides an alcohol concentration rapid detection stick monitoring and early warning system based on face detection function. In view of the problems that traditional alcohol concentration rapid detection sticks have a single application direction, cannot upload alcohol detection results to the cloud in real time, and the same driver is detected multiple times, which is not convenient for accurately calculating the inspection workload of on-duty police officers, this scheme combines the alcohol concentration rapid detection stick with the face detection function to avoid unnecessary repetitive detection, improve the work efficiency of inspectors, and realize data integration; in view of the problems that traditional fatigue driving methods have high computational complexity and low evaluation accuracy, this scheme adopts a multi-level adaptive 3D-CNN model to learn the driver's status to improve the performance of identifying fatigue driving; in view of the poor readability of traditional target detection methods and the inability to perform face recognition in a hierarchical and step-by-step manner, this scheme adopts a three-stage model with non-intersecting joint suppression technology to drive overall efficiency.

[0004] The present invention provides an alcohol concentration rapid detection stick monitoring and early warning system based on face detection function, comprising an alcohol concentration rapid detection stick terminal and a cloud server; the alcohol concentration rapid detection stick terminal comprises a camera acquisition module, an alcohol detection module, a communication module and a display module; the cloud server comprises a fatigue monitoring module and a face matching module, the camera acquisition module is provided with an industrial camera; the face matching module is provided with a driver information database; the driver information database contains basic information and historical records;

[0005] The camera acquisition module acquires images of the driver through an industrial camera, outputs an image frame sequence B and transmits it to the communication module;

[0006] The alcohol detection module detects whether the driver's alcohol concentration exceeds the standard and whether there is drunk driving, outputs the drunk driving detection result and transmits it to the communication module;

[0007] The communication module transmits the drunk driving detection result and the image frame sequence B to the fatigue monitoring module and the face matching module respectively;

[0008] The fatigue monitoring module receives the image frame sequence B from the communication module, detects whether the driver has fatigue driving behavior, and outputs the fatigue driving result to the face matching module;

[0009] The face matching module receives the image frame sequence B from the communication module, extracts face features and performs face matching, updates the fatigue driving results and the drunk driving detection results to the driver information database, and transmits the basic information and historical records, fatigue driving results and drunk driving detection results of the corresponding driver in the driver information database to the display module;

[0010] The display module receives and displays basic information, historical records, fatigue driving results and drunk driving detection results from the face matching module.

[0011] Furthermore, the camera acquisition module specifically includes the following contents:

[0012] Collect image frame sequence A through an industrial camera;

[0013] The image frame sequence A is resized to generate image Gaussian pyramids with different resolutions;

[0014] For target detection, 2D-CNN is used as the model in the target detection stage. It is trained and classified into two categories to determine whether the image frame sequence A contains the driver's face area. The cross entropy loss function of the model in the target detection stage is calculated. The formula is as follows:

[0015]

[0016] In the formula, Represents the cross entropy loss function of the model in the target detection stage, represents the expected binary classification result, P i represents the probability that the image frame sequence A contains a face area,

[0017] Regression bounding box, segmentation bounding box of face area, adopt three-stage architecture, process image Gaussian pyramid in cascade mode, the specific steps are as follows:

[0018] In the I-Net stage, 2D-CNN is used as the model of the I-Net stage. It is trained and regresses the face area. The filter size is fixed to 3x3. The overlapping bounding boxes containing the position and size information of the face area are output. The Euclidean loss function of the model in the regression bounding box stage is calculated. The formula is as follows:

[0019]

[0020] In the formula, represents the Euclidean loss function of the model at the bounding box regression stage, represents the actual bounding box, represents the expected bounding box, R 4 represents the four-dimensional real number space, represents the square of L2 norm;

[0021] P-Net stage: The bounding box is calibrated by NIOUS technology, and the overlapping bounding boxes are processed by non-maximum suppression to generate a merged bounding box;

[0022] O-Net stage: According to the merged bounding box, the left eye, right eye, nose, left end of the mouth and right end of the mouth are located to obtain the precise bounding box, and the image frame sequence A is cropped according to the precise bounding box to output the image frame sequence B;

[0023] The image frame sequence B is transmitted to the fatigue monitoring module.

[0024] Furthermore, the fatigue monitoring module specifically includes the following contents:

[0025] Spatiotemporal learning, using 3D-CNN as the model of the spatiotemporal learning stage, the image frame sequence B is input into 3D-CNN, and the spatial and temporal data is extracted. The forward propagation formula of 3D-CNN is as follows:

[0026]

[0027] In the formula, st represents time-space data, represents the spatial-temporal data of the output feature map of the i-th layer at the time of m rows, n columns and t, α represents the activation function, W represents the width, H represents the height, D represents the time, and x abc Represents the element at the ath row, bth column and cth moment in the image frame sequence B, Indicates the jth weight matrix of the i-th layer in x abc The weight coefficient of Indicates x abc and Perform 3D convolution operation, B ij represents the jth bias item of the i-th layer;

[0028] State understanding, in the form of one-hot coding, records the driver's glasses state, eye position state, yawning state, nodding state and eyebrow raising state, and defines the glasses state as O gn , define the eye position state as O e , the yawning state is defined as O m , define the nodding state as O h, the eyebrow raising state is defined as O sc , the status condition data is composed of O gn , O e , O m , O h and O sc Composition, using FCNN as the model of the state understanding stage, extracting state condition data, using Euclidean distance as the loss function of the model of the state understanding stage, the formula is as follows:

[0029]

[0030] In the formula, represents the loss function of the model in the state understanding stage, represents the expected candidate box, R 10 represents the ten-dimensional real number space, represents the actual candidate box, represents the square of L2 norm;

[0031] Back propagation optimizes the three-stage models, including the model of the state understanding stage, the model of the target detection stage, and the model of the state understanding stage, minimizing the total loss function. The objective function formula of back propagation is as follows:

[0032]

[0033] In the formula, min means minimization, N means the number of samples used to train the three-stage models, represents the sum of the losses of all samples used to train the three-stage model, ∑ j ∈{det,box,loc} represents the cumulative sum of the loss functions of the three-stage model, δ j Represents the weight coefficient of loss type j, Represents an indicator that determines whether the loss function of the i-th sample and the j-th stage model should be activated. represents the loss function of the i-th sample and the j-th stage model, represents the total loss function;

[0034] Feature fusion,FCNN is used as the model in the feature fusion stage to identify the adaptive conditional feature representation set. The calculation formula is as follows:

[0035] γ=W fu (W fea W gn O gn ⊙W h O h ⊙W m O m ⊙W eO e ⊙W sc O sc )+b fu ;

[0036] In the formula, γ represents the adaptive conditional feature representation set, W fu represents the fusion weight, b fu represents the fusion bias, W fea represents feature weight, ⊙ represents element-by-element multiplication, W gn Indicates O gn The associated parameter, W h Indicates O h The associated parameter, W m Indicates O m The associated parameter, W e Indicates O e The associated parameter, W sc Indicates O sc The associated parameters of

[0037] Fatigue detection,FCNN is used as the model of the fatigue detection stage to calculate the conditional feature clusters. The formula is as follows:

[0038]

[0039] In the formula, v i represents the i-th conditional feature cluster, γ j is the jth element of the adaptive conditional feature representation set, exp() represents the exponential operation, ∑ j exp(γ j ) means all exp(γ j )

[0040] Use conditional feature clusters for fatigue detection, the formula is as follows:

[0041] R Detect =lm detect (v,θ detect );

[0042] In the formula, R Detect Indicates the result of judging whether the driver is tired, lm detect () represents the model of fatigue detection stage, v represents the conditional feature cluster, θ detect Parameters of the model representing the fatigue detection phase.

[0043] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0044] (1) In view of the problems that traditional alcohol concentration rapid detection sticks have a single application direction, cannot upload alcohol test results to the cloud in real time, and the same driver is tested multiple times, data is repeated and redundant, and it is not convenient to accurately calculate the inspection workload of on-duty police officers. This solution combines the alcohol concentration rapid detection stick with the face detection function. While detecting drunk driving, it monitors whether the driver is driving fatigued, and synchronizes the test results to the server. Through the face recognition method, the test results are matched one by one with the driver's historical data, realizing the functions of automatic upload and data update, avoiding unnecessary repeated detection, improving the work efficiency of inspectors, and realizing data integration;

[0045] (2) To address the problems of high computational complexity and low evaluation accuracy of traditional fatigue driving methods, this solution introduces the time dimension, fully utilizes the video dynamic information contained in continuous image frames, and uses a multi-level adaptive 3D-CNN model to learn the driver's status to improve the performance of fatigue driving identification;

[0046] (3) To address the problem that traditional target detection methods have poor readability and cannot perform face recognition in a hierarchical and step-by-step manner, this scheme adopts a three-stage model with non-intersecting joint suppression technology. The I-Net, P-Net and O-Net stages perform target detection in the driver's face area, improving the flexibility of detection and driving overall efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic diagram of an alcohol concentration rapid detection stick monitoring and early warning system based on face detection function provided by the present invention.

[0048] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0050] In the description of the present invention, it should be understood that terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “top”, “bottom”, “inside” and “outside” indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0051] Example 1, see Figure 1 The present invention provides a rapid alcohol concentration detection stick monitoring and early warning system based on face detection function, including a rapid alcohol concentration detection stick terminal and a cloud server; the rapid alcohol concentration detection stick terminal includes a camera acquisition module, an alcohol detection module, a communication module and a display module; the cloud server includes a fatigue monitoring module and a face matching module, the camera acquisition module is provided with an industrial camera; the face matching module is provided with a driver information database; the driver information database contains basic information and historical records;

[0052] The camera acquisition module acquires images of the driver through an industrial camera, outputs an image frame sequence B and transmits it to the communication module;

[0053] The alcohol detection module detects whether the driver's alcohol concentration exceeds the standard and whether there is drunk driving, outputs the drunk driving detection result and transmits it to the communication module;

[0054] The communication module transmits the drunk driving detection result and the image frame sequence B to the fatigue monitoring module and the face matching module respectively;

[0055] The fatigue monitoring module receives the image frame sequence B from the communication module, detects whether the driver has fatigue driving behavior, and outputs the fatigue driving result to the face matching module;

[0056] The face matching module receives the image frame sequence B from the communication module, extracts face features and performs face matching, updates the fatigue driving results and the drunk driving detection results to the driver information database, and transmits the basic information and historical records, fatigue driving results and drunk driving detection results of the corresponding driver in the driver information database to the display module;

[0057] The display module receives and displays basic information, historical records, fatigue driving results and drunk driving detection results from the face matching module.

[0058] Example 2, see Figure 1 , this embodiment is based on the above embodiment, and the camera acquisition module specifically includes the following contents:

[0059] Collect image frame sequence A through an industrial camera;

[0060] The image frame sequence A is resized to generate image Gaussian pyramids with different resolutions;

[0061] For target detection, 2D-CNN is used as the model in the target detection stage. It is trained and classified into two categories to determine whether the image frame sequence A contains the driver's face area. The cross entropy loss function of the model in the target detection stage is calculated. The formula is as follows:

[0062]

[0063] In the formula, Represents the cross entropy loss function of the model in the target detection stage, represents the expected binary classification result, P i represents the probability that the image frame sequence A contains a face area,

[0064] Regression bounding box, segmentation bounding box of face area, adopt three-stage architecture, process image Gaussian pyramid in cascade mode, the specific steps are as follows:

[0065] In the I-Net stage, 2D-CNN is used as the model of the I-Net stage. It is trained and regresses the face area. The filter size is fixed to 3x3. The overlapping bounding boxes containing the position and size information of the face area are output. The Euclidean loss function of the model in the regression bounding box stage is calculated. The formula is as follows:

[0066]

[0067] In the formula, represents the Euclidean loss function of the model at the bounding box regression stage, represents the actual bounding box, represents the expected bounding box, R 4 represents the four-dimensional real number space, represents the square of L2 norm;

[0068] P-Net stage: The bounding box is calibrated by NIOUS technology, and the overlapping bounding boxes are processed by non-maximum suppression to generate a merged bounding box;

[0069] O-Net stage: According to the merged bounding box, the left eye, right eye, nose, left end of the mouth and right end of the mouth are located to obtain the precise bounding box, and the image frame sequence A is cropped according to the precise bounding box to output the image frame sequence B;

[0070] The image frame sequence B is transmitted to the fatigue monitoring module.

[0071] Example 3, see Figure 1, the fatigue monitoring module specifically includes the following contents:

[0072] Spatiotemporal learning, using 3D-CNN as the model of the spatiotemporal learning stage, the image frame sequence B is input into 3D-CNN, and the spatial and temporal data is extracted. The forward propagation formula of 3D-CNN is as follows:

[0073]

[0074] In the formula, st represents time-space data, represents the spatial-temporal data of the output feature map of the i-th layer at the time of m rows, n columns and t, α represents the activation function, W represents the width, H represents the height, D represents the time, and x abc Represents the element at the ath row, bth column and cth moment in the image frame sequence B, Indicates the jth weight matrix of the i-th layer in x abc The weight coefficient of Indicates x abc and Perform 3D convolution operation, B ij represents the jth bias item of the i-th layer;

[0075] State understanding, in the form of one-hot coding, records the driver's glasses state, eye position state, yawning state, nodding state and eyebrow raising state, and defines the glasses state as O gn , define the eye position state as O e , the yawning state is defined as O m , define the nodding state as O h , the eyebrow raising state is defined as O sc , the status condition data is composed of O gn , O e , O m , O h and O sc Composition, using FCNN as the model of the state understanding stage, extracting state condition data, using Euclidean distance as the loss function of the model of the state understanding stage, the formula is as follows:

[0076]

[0077] In the formula, represents the loss function of the model in the state understanding stage, represents the expected candidate box, R 10 represents the ten-dimensional real number space, represents the actual candidate box, represents the square of L2 norm;

[0078] Back propagation optimizes the three-stage models, including the model of the state understanding stage, the model of the target detection stage, and the model of the state understanding stage, minimizing the total loss function. The objective function formula of back propagation is as follows:

[0079]

[0080] In the formula, min means minimization, N means the number of samples used to train the three-stage models, represents the sum of the losses of all samples used to train the three-stage model, ∑ j ∈{det,box,loc} represents the cumulative sum of the loss functions of the three-stage model, δ j Represents the weight coefficient of loss type j, Represents an indicator that determines whether the loss function of the i-th sample and the j-th stage model should be activated. represents the loss function of the i-th sample and the j-th stage model, represents the total loss function;

[0081] Feature fusion,FCNN is used as the model in the feature fusion stage to identify the adaptive conditional feature representation set. The calculation formula is as follows:

[0082] γ=W fu (W fea W gn O gn ⊙W h O h ⊙W m O m ⊙W e O e ⊙W sc O sc )+b fu ;

[0083] In the formula, γ represents the adaptive conditional feature representation set, W fu represents the fusion weight, b fu represents the fusion bias, W fea represents feature weight, ⊙ represents element-by-element multiplication, W gn Indicates O gn The associated parameter, W h Indicates O h The associated parameter, W m Indicates O m The associated parameter, W e Indicates O e The associated parameter, W sc Indicates O sc The associated parameters of

[0084] Fatigue detection,FCNN is used as the model of the fatigue detection stage to calculate the conditional feature clusters. The formula is as follows:

[0085]

[0086] In the formula, v i represents the i-th conditional feature cluster, γ j is the jth element of the adaptive conditional feature representation set, exp() represents the exponential operation, ∑ j exp(γ j ) means all exp(γ j )

[0087] Use conditional feature clusters for fatigue detection, the formula is as follows:

[0088] R Detect =lm detect (v,θ detect );

[0089] In the formula, R Detect Indicates the result of judging whether the driver is tired, lm detect () represents the model of fatigue detection stage, v represents the conditional feature cluster, θ detect Parameters of the model representing the fatigue detection phase.

[0090] Example 4, see Figure 1 This embodiment is based on the above embodiment. In the fatigue monitoring module, α represents an activation function, specifically, ReLU function is used as the activation function of the model in the spatiotemporal learning stage.

[0091] Example 5, see Figure 1 This embodiment is based on the above embodiment. In the fatigue monitoring module, Represents an indicator that determines whether the i-th sample and the j-th loss type should be activated. Specifically, if no face is recognized in the i-th sample, that is, there is no target to be detected, then The loss function of the corresponding j-th loss type is fixed to 0, and there is no need to calculate the loss. If the face in the i-th sample is recognized, that is, the target to be detected, the loss function of the corresponding j-th loss type needs to be calculated.

[0092] Example 6, see Figure 1 This embodiment is based on the above embodiment. In the communication module, the WebSocket two-way communication protocol is used to realize data transmission between the alcohol concentration rapid detection stick terminal and the cloud server.

[0093] By executing the above operations, in order to address the problems of traditional traffic timing schemes, such as low timing scheme accuracy due to lack of real-time data support, congestion due to lack of intersection coordination, and long traffic recovery time due to lack of flexibility, this scheme takes into account multiple factors, controls traffic flow based on signal cycle and loss time, improves traffic efficiency, adjusts signal timing schemes in real time based on lane capacity, improves flexibility, minimizes the average delay of vehicles at intersections based on the objective function, and improves overall traffic efficiency and service level.

[0094] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0095] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0096] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A rapid alcohol concentration detection stick monitoring and early warning system based on face detection function, characterized in that: It includes an alcohol concentration rapid detection stick terminal and a cloud server. The alcohol concentration rapid detection stick terminal includes a camera acquisition module, an alcohol detection module, a communication module and a display module; the cloud server includes a fatigue monitoring module and a face matching module. The camera acquisition module is provided with an industrial camera; the face matching module is provided with a driver information database; the driver information database contains basic information and historical records; The camera acquisition module acquires images of the driver through an industrial camera, outputs an image frame sequence B and transmits it to the communication module; The alcohol detection module detects whether the driver's alcohol concentration exceeds the standard and whether there is drunk driving, outputs the drunk driving detection result and transmits it to the communication module; The communication module transmits the drunk driving detection result and the image frame sequence B to the fatigue monitoring module and the face matching module respectively; The fatigue monitoring module receives the image frame sequence B from the communication module, detects whether the driver has fatigue driving behavior, and outputs the fatigue driving result to the face matching module; The face matching module receives the image frame sequence B from the communication module, extracts face features and performs face matching, updates the fatigue driving results and the drunk driving detection results to the driver information database, and transmits the basic information and historical records, fatigue driving results and drunk driving detection results of the corresponding driver in the driver information database to the display module; The display module receives and displays basic information, historical records, fatigue driving results and drunk driving detection results from the face matching module; The camera acquisition module comprises the following steps: Collect image frame sequence A through an industrial camera; The image frame sequence A is resized to generate image Gaussian pyramids with different resolutions; For target detection, 2D-CNN is used as the model in the target detection stage. It is trained and classified into two categories to determine whether the image frame sequence A contains the driver's face area. The cross entropy loss function of the model in the target detection stage is calculated. The formula is as follows: ； In the formula, Represents the cross entropy loss function of the model in the target detection stage, represents the expected binary classification result, represents the probability that the image frame sequence A contains a face area, ; Regression bounding box, segmentation bounding box of face area, adopt three-stage architecture, process image Gaussian pyramid in cascade mode, the specific steps are as follows: In the I-Net stage, 2D-CNN is used as the model of the I-Net stage. It is trained and regresses the face area. The filter size is fixed to 3x3. The overlapping bounding boxes containing the position and size information of the face area are output. The Euclidean loss function of the model in the regression bounding box stage is calculated. The formula is as follows: ； In the formula, represents the Euclidean loss function of the model at the bounding box regression stage, represents the actual bounding box, represents the expected bounding box, , represents the four-dimensional real number space, represents the square of L2 norm; P-Net stage: The bounding box is calibrated by NIOUS technology, and the overlapping bounding boxes are processed by non-maximum suppression to generate a merged bounding box; O-Net stage: According to the merged bounding box, the left eye, right eye, nose, left end of the mouth and right end of the mouth are located to obtain the precise bounding box, and the image frame sequence A is cropped according to the precise bounding box to output the image frame sequence B; Transmitting the image frame sequence B to the fatigue monitoring module; The fatigue monitoring module comprises the following steps: Spatiotemporal learning, using 3D-CNN as the model of the spatiotemporal learning stage, the image frame sequence B is input into 3D-CNN, and the spatial and temporal data is extracted. The forward propagation formula of 3D-CNN is as follows: ； In the formula, Represents time-space data, Indicates the output Tier The feature map is wide Line Ledi The spatial and temporal data of the moment, represents the activation function, Indicates width, Indicates height, Indicates time, Represents the first Line Ledi The elements of time, Indicates Tier The weight matrix is The weight coefficient of Express and Perform 3D convolution operation, Indicates Tier Bias terms; State understanding, in the form of one-hot encoding, records the driver's glasses state, eye position state, yawning state, nodding state and eyebrow raising state, and defines the glasses state as , the eye position state is defined as , the yawning state is defined as , define the nodding state as , the eyebrow raising state is defined as , the state condition data is given by , , , and Composition, using FCNN as the model of the state understanding stage, extracting state condition data, using Euclidean distance as the loss function of the model of the state understanding stage, the formula is as follows: ； In the formula, represents the loss function of the model in the state understanding stage, represents the expected candidate box, , represents the ten-dimensional real number space, represents the actual candidate box, represents the square of L2 norm; Back propagation optimizes the three-stage models, including the model of the state understanding stage, the model of the target detection stage, and the model of the state understanding stage, minimizing the total loss function. The objective function formula of back propagation is as follows: ； In the formula, represents minimization, represents the number of samples used to train the three-stage model, represents the sum of the losses of all samples used to train the three-stage models, represents the cumulative sum of the loss functions of the three stage models, Indicates the loss type is The weight coefficient of Indicates confirmation Samples and An indicator of whether the loss function of the model at this stage should be activated. Indicates Samples and The loss function of the model in this stage is: represents the total loss function; Feature fusion,FCNN is used as the model in the feature fusion stage to identify the adaptive conditional feature representation set. The calculation formula is as follows: ； In the formula, represents the adaptive conditional feature representation set, represents the fusion weight, represents the fusion bias, represents the feature weight, represents element-wise multiplication, express The associated parameters, express The associated parameters, express The associated parameters, express The associated parameters, express The associated parameters of Fatigue detection,FCNN is used as the model of the fatigue detection stage to calculate the conditional feature clusters. The formula is as follows: ； In the formula, Indicates conditional feature clusters, is the first of the adaptive conditional feature representation sets elements, represents exponential operation, Indicates all The cumulative sum of Use conditional feature clusters for fatigue detection, the formula is as follows: ； In the formula, Indicates the result of judging whether the driver is tired. A model representing the fatigue detection phase, represents the conditional feature cluster, Parameters of the model representing the fatigue detection phase.

Images