A driver status analysis and judgment method based on deep learning neural network

Through deep learning neural network and adaptive driving scenario generation model, the existing driver's state detection methods have solved the problems of expensive equipment, low detection accuracy and poor model robustness, and efficient and robust driver's state analysis is achieved. Enhanced detection diversity and reliability.

CN115909296BActive Publication Date: 2024-11-05WUXI HUIYAN ARTIFICIAL INTELLIGENCE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211645063.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-12-20
Publication Date
2024-11-05
Estimated Expiration
2042-12-20

AI Technical Summary

Technical Problem

The existing driver status detection methods rely on physiological feature monitoring, motion feature monitoring or facial feature monitoring, and there are problems such as expensive equipment, easy to damage, low detection accuracy and poor model robustness. In particular, facial information-dependent methods rely heavily on training. Sample sets lead to large algorithm deviations.

Method used

Using the driver's state analysis and judgment method based on deep learning neural network, the adaptive driving scenario generation model is constructed, and the hidden space coded vector is extracted from the driving state image using the encoding model and the analytical model, and the driving state image of multiple scenes is generated. The improved L-BFGS algorithm optimizes model parameters to improve the robustness and iterative efficiency of the model.

Benefits of technology

It realizes the robustness and accuracy of driver status analysis without relying on expensive equipment, reduces the impact on external factors, enhances the diversity of driving scenarios, and improves the reliability of driver status detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115909296B_ABST
    Figure CN115909296B_ABST
Patent Text Reader

Abstract

The present invention relates to the technical field of driver state analysis, and discloses a driver state analysis and judgment method based on a deep learning neural network, the method comprising: constructing an adaptive driving scene generation model and determining an adaptive driving scene generation optimization objective function; optimizing and solving the objective function using an improved L‑BFGS algorithm to obtain an optimal model, and inputting images in a driving state image set into the model to generate a large number of driving state images, training a driver state analysis neural network, and using the trained driver state analysis neural network to perform driver state analysis and judgment. The present invention uses a coding model to extract a latent space coding vector with noise from an original image, and then reconstructs the extracted vector into a generated image to achieve multi-scenario driving state image generation, and uses a multi-particle improved L‑BFGS algorithm to optimize and solve the adaptive driving scene generation model parameters to quickly obtain the optimal model parameters.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of driver status analysis, and in particular to a driver status analysis and judgment method based on a deep learning neural network. Background Art

[0002] In recent years, traffic accidents have occurred frequently, causing a lot of losses to society. Traffic accidents caused by fatigue driving lights and drivers' inattention have increased year by year. For truck drivers, the driver's state affects their personal safety. At present, driver state detection and analysis mainly include three types of methods: monitoring the driver's physiological characteristics; monitoring based on the vehicle's motion characteristics; monitoring the driver's facial features. Judging whether the driver is fatigued by physiological characteristics requires wearing monitoring equipment, which will affect the driver's driving experience, and the equipment is expensive and easy to damage. Motion feature monitoring mainly judges the fatigue state by monitoring the vehicle's physical information, such as the vehicle's driving trajectory, sway amplitude, and hand pressure on the steering wheel. Although this method does not affect the driver's driving experience, it will be affected by some external factors such as the driver's driving habits and special road conditions. Generally, it will not be used as the main basis for judgment. Most of the current driver state detection methods are based on facial information for fatigue detection. However, such methods rely heavily on training sample sets, resulting in large algorithm deviations, low detection accuracy, and poor model robustness. To address this problem, this patent proposes a driver status analysis and judgment method based on a deep learning neural network, which enhances the diversity of driving scenarios through adaptive sample expansion and improves the robustness of the driver's driving status analysis. Summary of the invention

[0003] In view of this, the present invention provides a driver state analysis and judgment method based on a deep learning neural network, the purpose of which is to: 1) construct an adaptive driving scene generation model based on a coding model and an analytical model, extract a latent space coding vector with noise from the pixel distribution of the collected driving state image, generate a multi-scene driving state image based on the latent space coding vector, and construct an adaptive driving scene generation optimization objective function, the optimization goal is to approximate the posterior probability distribution closer to the real posterior probability distribution, and ensure that the extracted pixel distribution is the effective pixel distribution of the real image; 2) adopt an improved L-BFGS algorithm of multiple particles to optimize and solve the parameters of the adaptive driving scene generation model, on the one hand, generate high-dimensional multiple particles, use the high-dimensional position coordinates of each particle to characterize the model parameters, expand the feasible solution range of the model parameters, and use the L-BFGS algorithm combined with the shrinkage coefficient and the speed parameter to update the position coordinates of each particle, speed up the iteration efficiency of the L-BFGS algorithm, when the gradient change rate of the particle is small, the position and speed of the particle are no longer updated, the number of particles is effectively reduced, and the rapid optimization and solution of the parameters of the adaptive driving scene generation model are realized, and then a usable model is obtained.

[0004] To achieve the above object, the present invention provides a method for analyzing and judging a driver's state based on a deep learning neural network, comprising the following steps:

[0005] S1: Collecting and preprocessing the driver's driving state images to obtain a standardized driving state image set;

[0006] S2: construct an adaptive driving scene generation model, which includes an encoding model and an analysis model. The encoding model takes the normalized driving state image as input and outputs the latent space encoding vector, and the analysis model takes the latent space encoding vector as input and outputs the generated multi-scenario driving state image.

[0007] S3: determining an adaptive driving scenario generation optimization objective function according to the constructed adaptive driving scenario generation model;

[0008] S4: Based on the collected driving state image set, the improved L-BFGS algorithm is used to optimize the adaptive driving scene generation optimization objective function to obtain the optimal model parameters;

[0009] S5: constructing an optimal adaptive driving scene generation model according to the optimal model parameters obtained by solving, and inputting images in the driving state image set into the model to generate a large number of driving state images;

[0010] S6: Using the collected and generated driving status images as a training set, training the driver status analysis neural network, and using the trained driver status analysis neural network to perform driver status analysis and judgment.

[0011] As a further improvement method of the present invention:

[0012] Optionally, in step S1, collecting the driver's driving state image and preprocessing it includes:

[0013] The driver's driving state image is collected during the driver's driving process, wherein the driver's driving state image is an image of the driver driving the vehicle in different scenes and at different angles;

[0014] The collected driver driving state images are preprocessed to obtain driver driving state images with the same image specifications and in grayscale, and all preprocessed driver driving state images constitute a driving state image set, wherein the preprocessing process is:

[0015] S11: finding the maximum value of the three RGB color components of each pixel in the driver's driving state image, and setting the maximum value as the grayscale value of the pixel to obtain a grayscale image of the driver's driving state image;

[0016] S12: performing image normalization processing on the grayscale image of the driver's driving state with any size of M×N pixels, wherein the formula for image normalization processing is:

[0017]

[0018] in:

[0019] (i, j) represents the pixel in the i-th row and j-th column of the grayscale image of the driver's driving state image with a size of M×N pixels, M represents the number of row pixels of the driver's driving state image, N represents the number of column pixels of the driver's driving state image, and (i′, j′) represents the pixel after normalization processing, that is, the pixel in the i-th row and j-th column is mapped to the i′th row and j′th column;

[0020] M′×N′ pixels represent the grayscale image specifications of the driver’s driving status image after normalization;

[0021] S13: Calculate the grayscale value average of the adjacent pixels of the empty pixel, and use the grayscale value average as the grayscale value of the empty pixel, wherein the empty pixel represents a pixel without a grayscale value in the grayscale image after normalization processing;

[0022] S14: stretching the grayscale value of each pixel in the normalized grayscale image:

[0023]

[0024] in:

[0025] Gray(i′, j′) represents the gray value of pixel (i′, j′) in the grayscale image after normalization;

[0026] MIN Gray It is the minimum grayscale value in the grayscale image after normalization;

[0027] MAX Gray It is the maximum grayscale value in the grayscale image after normalization;

[0028] g(i′, j′) is the grayscale value of pixel (i′, j′) after stretching.

[0029] Optionally, constructing the adaptive driving scenario generation model in step S2 includes:

[0030] Constructing an adaptive driving scene generation model, wherein the adaptive driving scene generation model includes an encoding model and a parsing model, wherein the encoding model takes a normalized driving state image as input and a corresponding latent space encoding vector as output, and the parsing model takes a latent space encoding vector as input and a generated multi-scenario driving state image as output;

[0031] The encoding model is a hidden layer containing multiple neurons, and the parsing model is a fully connected neural network;

[0032] The process of generating multi-scenario driving state images based on the adaptive driving scene generation model is as follows:

[0033] S21: Calculate the pixel distribution x of the input k-th normalized driving state image k :

[0034] x k ={p k,L |L∈[0,255]}

[0035]

[0036] in:

[0037] P k,L represents the probability distribution of the gray level L in the k-th normalized driving state image, count(L, k) represents the number of pixels with gray value L in the k-th normalized driving state image, k∈[1, K];

[0038] S22: Input the pixel distribution xk into the coding model:

[0039]

[0040] μ k =w2[tanh(w1x k +b1)]+b2

[0041]

[0042] in:

[0043] q(z|x k ) represents the approximate posterior probability distribution, z represents the latent space encoding vector distribution;

[0044] It means that given any x k , z all obey the multivariate Gaussian distribution independent of each dimension I represents the identity matrix, μ k , represents z|x k The Gaussian distribution parameters are obeyed. In the embodiment of the present invention, the dimension of the Gaussian distribution parameters is 256, which represents the Gaussian distribution parameters at different gray levels;

[0045] w1, w2, w3 represent the weight coefficients of the hidden layer in the encoding model, b1, b2, b3 represent the bias of the hidden layer in the encoding model;

[0046] S23: Coding model based on the solved Gaussian distribution parameter μ k , Generate x k The latent space encoding vector of :

[0047] z k,s =μ k +σ k ⊙ε s , ε s ~N(0,1)

[0048] in:

[0049] z k,s represents the generated noise ε added s x k The latent space encoding vector, ε s ~N(0,1) represents the noise ε s It conforms to the Gaussian distribution, s∈[1,S], S represents the number of types of noise added; ⊙ represents element-by-element multiplication;

[0050] S24: Input the latent space encoding vector into the parsing model to obtain the true posterior probability distribution p(x k |z k,s ):

[0051] log p(x k |z k,s )=x k log z′ k,s +(1-x k )log(1-z′ k,s )

[0052] z′ k,s =ReLU[w5tanh(w4z k,s +b4)+b5]

[0053] in:

[0054] w4, w5 represent the weight coefficients of the hidden layer in the analytical model, b4, b5 represent the bias of the hidden layer in the analytical model;

[0055] ReLU(·) represents the activation function;

[0056] From the true posterior probability distribution p(x k |z k,s ) Extract the distribution parameter μ′ k,s , (σ′ k,s ) 2 Generate image pixel distribution x k (s), where x k (s) = μ′ k,s =w5tanh(w4z k,s +b4)+b5, pixel distribution x k The standard deviation σ′ of (s) k,s is log(σ′ k,s ) 2 =ReLU[w5tanh(w4z k,s +b4)+b5];

[0057] S25: Based on image pixel distribution x k (s) Generate driving status image.

[0058] Optionally, determining the adaptive driving scenario generation optimization objective function in step S3 includes:

[0059] An adaptive driving scene generation optimization objective function is determined according to the constructed adaptive driving scene generation model, wherein the optimization goal of the adaptive driving scene generation optimization objective function is to make the approximate posterior probability distribution closer to the true posterior probability distribution and generate a more effective latent space encoding vector;

[0060] The determined adaptive driving scenario generation optimization objective function F(θ1, θ2) is:

[0061]

[0062] in:

[0063] represents the approximate posterior probability distribution q(z|x k ) and the true posterior probability distribution p(x k |z), where z represents the distribution of latent space encoding vectors;

[0064] θ1, θ2 represent the encoding model parameters and parsing model parameters respectively;

[0065] When the latent space encoding vector distribution z and the approximate posterior probability distribution q(z|x k ) are consistent, the KL divergence is minimized, and the KL divergence is simplified based on the output of the encoding model:

[0066]

[0067] in:

[0068] L represents the grayscale range, μ k (L), σ k (L) represents the Gaussian distribution parameter in L dimension, which is the output result of the coding model based on parameter θ1;

[0069] And simplify the true posterior probability distribution in the function to the L1 norm:

[0070]

[0071] The simplified adaptive driving scenario generation optimization objective function F′(θ1, θ2) is:

[0072]

[0073] in:

[0074] ||·|| denotes the L1 norm.

[0075] Optionally, in step S4, based on the collected driving state image set, an improved L-BFGS algorithm is used to optimize and solve the adaptive driving scene generation optimization objective function, including:

[0076] Based on the collected driving state image set, the improved L-BFGS algorithm is used to optimize the adaptive driving scene generation optimization objective function to obtain the optimal model parameters, wherein the model parameters to be optimized include the encoding model parameters θ1 and the analytical model parameters θ2, the encoding model parameters include the weight coefficients and biases of the hidden layers in the encoding model, and the analytical model parameters include the weight coefficients and biases of the hidden layers in the analytical model;

[0077] The optimization process of adaptive driving scenario generation optimization objective function is as follows:

[0078] S41: Initialize and generate n particles, each particle has a coordinate dimension of 10, corresponding to 10 model parameters to be optimized, where the initial coordinates loc(d0) and velocity v(d0) of any d-th particle are:

[0079]

[0080]

[0081] in:

[0082] represents the initial coordinates of the dth particle in the 10th dimension, The update speed of the coordinates corresponding to the dimension;

[0083] S42: Set the current iteration number of the optimization solution algorithm to t and the maximum iteration number to Max. Then the coordinate value of any d-th particle in the t-th iteration is loc(d t ), the corresponding speed is v(d t ), where the initial value of t is 1;

[0084] S43: At the beginning of each iteration, the coordinates of the n particles are respectively constructed as a coding model and a decoding model, and the driving state image set is input into the constructed coding model and decoding model to obtain the adaptive driving scene generation optimization objective function value F′(loc(d t )), where F′(loc(d t )) represents the optimization objective function value of the adaptive driving scenario generation based on the coordinate value of the d-th particle of the t-th iteration;

[0085] S44: Update the position coordinates of each particle using the L-BFGS algorithm:

[0086]

[0087]

[0088]

[0089] in:

[0090] I is the identity matrix;

[0091] a t-1 =loc(d t )-loc(d t-1 ), a0=d0;

[0092]

[0093] D0 is the identity matrix;

[0094] Indicates the calculation of gradient;

[0095] β1 represents the initial contraction coefficient, β1 = 1;

[0096] β2 represents the final shrinkage coefficient, β2 = 0.5;

[0097] S45: Update the velocity of each particle:

[0098]

[0099] in:

[0100] α represents the learning rate, which is set to 0.2. It represents the speed of the particle with the smallest gradient change rate at the t-1th iteration after the position coordinates are updated, where the gradient change rate of the dth particle after the position coordinates are updated is

[0101] S46: Calculate the gradient change rate of each particle after the tth iteration. If there is a particle whose gradient change rate is less than the preset threshold ∈, then end the update iteration of the particle position and velocity, set t=t+1, and return to step S43;

[0102] S47: When all particles have finished iteration or reached the maximum number of iterations Max, the adaptive driving scene generation optimization objective function value of each particle is calculated, and the particle position with the smallest adaptive driving scene generation optimization objective function value is selected as the optimal model parameter obtained by solving.

[0103] Optionally, in step S5, constructing an optimal adaptive driving scene generation model according to the optimal model parameters obtained by solving, includes:

[0104] A coding model and a parsing model are constructed according to the solved weight coefficients and the bias, and the constructed coding model and the parsing model are used as the constructed optimal adaptive driving scene generation model.

[0105] Optionally, in step S5, inputting images in the driving state image set into the model to generate a large number of driving state images includes:

[0106] The images in the driving state image set are input into the encoding model of the optimal adaptive driving scene generation model to obtain the Gaussian distribution parameters of each image, and S different noises are added to the Gaussian distribution parameters of each image in turn to generate S latent space encoding vectors, and the S latent space encoding vectors are input into the parsing model in turn to obtain S generated images of the input image.

[0107] Optionally, in step S6, the collected and generated driving state images are used as a training set to train the driver state analysis neural network, including:

[0108] Constructing a driver state analysis neural network, wherein the driver state analysis neural network comprises an encoding layer, a facial feature detection layer and a state analysis layer. The input of the encoding layer is a driving state image. The driving state image is convolutionally encoded using a convolution module, and the encoding result is input into the facial feature detection layer. The structure of the facial feature detection layer is a YOLOv3 model. The facial feature regions of the driver are marked using the facial feature detection layer, and the mouth opening and eye opening are calculated in the state analysis layer to determine the driver's fatigue state. When the eye opening in the driver's driving state image sequence is mostly lower than a preset threshold value ω1, and the mouth opening is mostly higher than a preset threshold value ω2, it indicates that the driver is in a fatigue state.

[0109] The collected and generated driving state images are used as training set data to train the driver state analysis neural network. The training objective function of the driver state analysis neural network is:

[0110]

[0111] in:

[0112] IOU c (m) represents the intersection-and-union ratio between the mth facial feature region obtained by marking the encoding result of the driving state image c in the training set data by the facial feature detection layer and the real facial feature region;

[0113] The driver state neural network is trained using the gradient descent method, and the trained driver state analysis neural network is used to analyze and judge the driver state.

[0114] In order to solve the above problem, the present invention provides an electronic device, the electronic device comprising:

[0115] A memory storing at least one instruction;

[0116] Communication interface, enabling electronic equipment to communicate; and

[0117] The processor executes the instructions stored in the memory to implement the above-mentioned driver status analysis and judgment method based on deep learning neural network.

[0118] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is executed by a processor in an electronic device to implement the above-mentioned driver status analysis and judgment method based on deep learning neural network.

[0119] Compared with the prior art, the present invention proposes a driver status analysis and judgment method based on a deep learning neural network, which has the following advantages:

[0120] First, this scheme proposes a multi-scenario driving state image generation model, by constructing an adaptive driving scene generation model, wherein the adaptive driving scene generation model includes a coding model and a parsing model, the coding model takes a normalized driving state image as input and a corresponding latent space coding vector as output, and the parsing model takes a latent space coding vector as input and a generated multi-scenario driving state image as output; the multi-scenario driving state image generation process based on the adaptive driving scene generation model is: calculate the pixel distribution xk of the input kth normalized driving state image:

[0121] x k ={p k,L |L∈[0,255]}

[0122]

[0123] Where: P k,L represents the probability distribution of gray level L in the k-th normalized driving state image, count(L, k) represents the number of pixels with gray value L in the k-th normalized driving state image, k∈[1, K]; the pixel distribution x k Input to the encoding model:

[0124]

[0125] μ k =w2[tanh(w1x k +b1)]+b2

[0126]

[0127] Where: q(z|x k ) represents the approximate posterior probability distribution, z represents the latent space encoding vector distribution; It means that given any x k , z all obey the multivariate Gaussian distribution independent of each dimension I represents the identity matrix, μ k , represents z|x k The Gaussian distribution parameters are obeyed; w1, w2, w3 represent the weight coefficients of the hidden layer in the coding model, b1, b2, b3 represent the bias of the hidden layer in the coding model; the coding model is based on the Gaussian distribution parameters μ obtained by solving k , Generate x k The latent space encoding vector of :

[0128] z k , s =μ k +σ k⊙)ε s , ε s ~N(0,1)

[0129] Where: z k,s represents the generated noise ε added s x k The latent space encoding vector, ε s ~N(0,1) represents the noise ε s It conforms to the Gaussian distribution, s∈[1,S], S represents the number of types of noise added; ⊙ represents element-by-element multiplication; the latent space encoding vector is input into the analytical model to obtain the true posterior probability distribution p(x k |z k,s ):

[0130] log p(x k |z k,s )=x k log z′ k,s +(1-x k )log(1-z′ k,s )

[0131] z′ k,s =ReLU[w5tanh(w4z k,s +b4)+b5]

[0132] Among them: w4, w5 represent the weight coefficients of the hidden layer in the analytical model, b4, b5 represent the bias of the hidden layer in the analytical model; ReLU(·) represents the activation function; from the true posterior probability distribution p(x k |z k,s ) Extract the distribution parameter μ′ k,s , (σ′ k,s ) 2 Generate image pixel distribution x k (s), where x k (s) = μ′ k,s =w5tanh(w4z k,s +b4)+b5, pixel distribution x k The standard deviation σ′ of (s) k,s is log(σ′ k,s ) 2 =ReLU[w5tanh(w4z k,s +b4)+b5]; based on the image pixel distribution x k(s) Generate a driving state image. The adaptive driving scene generation optimization objective function is determined according to the constructed adaptive driving scene generation model, wherein the optimization goal of the adaptive driving scene generation optimization objective function is to make the approximate posterior probability distribution closer to the true posterior probability distribution and generate a more effective latent space encoding vector; the determined adaptive driving scene generation optimization objective function F(θ1, θ2) is:

[0133]

[0134] in: represents the approximate posterior probability distribution q(z|x k ) and the true posterior probability distribution p(x k |z), z represents the distribution of latent space coding vectors; θ1 and θ2 represent the coding model parameters and the analytical model parameters respectively. This scheme constructs an adaptive driving scene generation model based on the coding model and the analytical model, extracts the latent space coding vector with noise from the pixel distribution of the collected driving state image, generates multi-scene driving state images based on the latent space coding vector, and constructs an adaptive driving scene generation optimization objective function. The optimization goal is to make the approximate posterior probability distribution closer to the true posterior probability distribution, and ensure that the extracted pixel distribution is the effective pixel distribution of the real image.

[0135] At the same time, this scheme proposes a fast solution process for the objective function. Initialize and generate n particles. The coordinate dimension of each particle is 10, corresponding to 10 model parameters to be optimized. The initial coordinates loc(d0) and velocity v(d0) of any dth particle are:

[0136]

[0137]

[0138] in: represents the initial coordinates of the dth particle in the 10th dimension, The update speed of the coordinates corresponding to the dimension; set the current iteration number of the optimization solution algorithm to t and the maximum iteration number to Max, then the coordinate value of any d-th particle in the t-th iteration is loc(d t ), the corresponding speed is v(d t ), where the initial value of t is 1; at the beginning of each iteration, the coordinates of the n particles are constructed as a coding model and a decoding model, and the driving state image set is input into the constructed coding model and decoding model to obtain the adaptive driving scene generation optimization objective function value F′(loc(d t )), where F′(loc(d t)) represents the adaptive driving scenario generation optimization objective function value of the model constructed based on the coordinate value of the d-th particle in the t-th iteration; the position coordinates of each particle are updated in combination with the L-BFGS algorithm:

[0139]

[0140]

[0141]

[0142] Where: I is the unit matrix; a t-1 =loc(d t )-loc(d t-1 ), a0=d0; D0 is the identity matrix; Indicates the calculated gradient; β1 indicates the initial contraction coefficient, β1 = 1; β2 indicates the final contraction coefficient, β2 = 0.5; the velocity of each particle is updated:

[0143]

[0144] Among them: α represents the learning rate, which is set to 0.2, It represents the speed of the particle with the smallest gradient change rate at the t-1th iteration after the position coordinates are updated, where the gradient change rate of the dth particle after the position coordinates are updated is Calculate the gradient change rate of each particle after the tth iteration. If there is a particle whose gradient change rate is less than the preset threshold ∈, then end the update iteration of the particle position and speed, set t = t + 1, and update the particle position and speed parameters; when all particles have finished iteration or reached the maximum number of iterations Max, calculate the adaptive driving scene generation optimization objective function value of each particle, and select the particle position with the smallest adaptive driving scene generation optimization objective function value as the optimal model parameter obtained by solution. This scheme uses the improved L-BFGS algorithm of multiple particles to optimize and solve the adaptive driving scene generation model parameters. On the one hand, high-dimensional multiple particles are generated, and the high-dimensional position coordinates of each particle are used to characterize the model parameters, which expands the feasible solution range of the model parameters. The position coordinates of each particle are updated by the L-BFGS algorithm combined with the contraction coefficient and the speed parameter, which speeds up the iteration efficiency of the L-BFGS algorithm. When the gradient change rate of the particle is small, the position and speed of the particle are no longer updated, which effectively reduces the number of particles, realizes the rapid optimization solution of the adaptive driving scene generation model parameters, and then obtains a usable model. BRIEF DESCRIPTION OF THE DRAWINGS

[0145] Figure 1A flowchart of a method for analyzing and judging a driver's state based on a deep learning neural network provided by an embodiment of the present invention;

[0146] Figure 2 A schematic diagram of the structure of an electronic device for implementing a method for analyzing and judging a driver's state based on a deep learning neural network provided in one embodiment of the present invention.

[0147] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0148] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0149] The embodiment of the present application provides a method for analyzing and judging a driver's state based on a deep learning neural network. The execution subject of the method for analyzing and judging a driver's state based on a deep learning neural network includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, the method for analyzing and judging a driver's state based on a deep learning neural network can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc.

[0150] Embodiment 1:

[0151] S1: Collect the driver's driving status images and preprocess them to obtain a standardized driving status image set.

[0152] The step S1 includes collecting the driver's driving state image and preprocessing it, including:

[0153] The driver's driving state image is collected during the driver's driving process, wherein the driver's driving state image is an image of the driver driving the vehicle in different scenes and at different angles;

[0154] The collected driver driving state images are preprocessed to obtain driver driving state images with the same image specifications and in grayscale, and all preprocessed driver driving state images constitute a driving state image set, wherein the preprocessing process is:

[0155] S11: finding the maximum value of the three RGB color components of each pixel in the driver's driving state image, and setting the maximum value as the grayscale value of the pixel to obtain a grayscale image of the driver's driving state image;

[0156] S12: performing image normalization processing on the grayscale image of the driver's driving state with any size of M×N pixels, wherein the formula for image normalization processing is:

[0157]

[0158] in:

[0159] (i, j) represents the pixel in the i-th row and j-th column of the grayscale image of the driver's driving state image with a size of M×N pixels, M represents the number of row pixels of the driver's driving state image, N represents the number of column pixels of the driver's driving state image, and (i′, j′) represents the pixel after normalization processing, that is, the pixel in the i-th row and j-th column is mapped to the i′th row and j′th column;

[0160] M′×N′ pixels represent the grayscale image specifications of the driver’s driving status image after normalization;

[0161] S13: Calculate the grayscale value average of the adjacent pixels of the empty pixel, and use the grayscale value average as the grayscale value of the empty pixel, wherein the empty pixel represents a pixel without a grayscale value in the grayscale image after normalization processing;

[0162] S14: stretching the grayscale value of each pixel in the normalized grayscale image:

[0163]

[0164] in:

[0165] Gray(i′, j′) represents the gray value of pixel (i′, j′) in the grayscale image after normalization;

[0166] MIN Gray It is the minimum grayscale value in the grayscale image after normalization;

[0167] MAX Gray It is the maximum grayscale value in the grayscale image after normalization;

[0168] g(i′, j′) is the grayscale value of pixel (i′, j′) after stretching.

[0169] S2: Construct an adaptive driving scene generation model, which includes a coding model and a parsing model. The coding model takes a normalized driving state image as input and outputs a latent space coding vector, and the parsing model takes a latent space coding vector as input and outputs a generated multi-scenario driving state image.

[0170] The step S2 constructs an adaptive driving scenario generation model, including:

[0171] Constructing an adaptive driving scene generation model, wherein the adaptive driving scene generation model includes an encoding model and a parsing model, wherein the encoding model takes a normalized driving state image as input and a corresponding latent space encoding vector as output, and the parsing model takes a latent space encoding vector as input and a generated multi-scenario driving state image as output;

[0172] The encoding model is a hidden layer containing multiple neurons, and the parsing model is a fully connected neural network;

[0173] The process of generating multi-scenario driving state images based on the adaptive driving scene generation model is as follows:

[0174] S21: Calculate the pixel distribution xk of the input k-th normalized driving state image:

[0175] x k ={p k,L |L∈[0,255]}

[0176]

[0177] in:

[0178] p k,L represents the probability distribution of the gray level L in the k-th normalized driving state image, count(L, k) represents the number of pixels with gray value L in the k-th normalized driving state image, k∈[1, K];

[0179] S22: Input the pixel distribution xk into the coding model:

[0180]

[0181] μ k =w2[tanh(w1x k +b1)]+b2

[0182]

[0183] in:

[0184] q(z|x k ) represents the approximate posterior probability distribution, z represents the latent space encoding vector distribution;

[0185] Indicates that given any xk, z obeys a multivariate Gaussian distribution independent of each dimension I represents the identity matrix, μ k , represents z|x k The Gaussian distribution parameters are obeyed. In the embodiment of the present invention, the dimension of the Gaussian distribution parameters is 256, which represents the Gaussian distribution parameters at different gray levels;

[0186] w1, w2, w3 represent the weight coefficients of the hidden layer in the encoding model, b1, b2, b3 represent the bias of the hidden layer in the encoding model;

[0187] S23: Coding model based on the solved Gaussian distribution parameter μ k , Generate x k The latent space encoding vector of :

[0188] z k,s =μ k +σ k ⊙ε s , ε s ~N(0,1)

[0189] in:

[0190] z k,s represents the generated noise ε added s x k The latent space encoding vector, ε s ~N(0,1) represents the noise ε s It conforms to the Gaussian distribution, s∈[1,S], S represents the number of types of noise added; ⊙ represents element-by-element multiplication;

[0191] S24: Input the latent space encoding vector into the parsing model to obtain the true posterior probability distribution p(x k |z k,s ):

[0192] log p(x k |z k,s )=x k log z′ k,s +(1-x k )log(1-z′ k,s )

[0193] z′ k,s =ReLU[w5tanh(w4z k,s +b4)+b5]

[0194] in:

[0195] w4, w5 represent the weight coefficients of the hidden layer in the analytical model, b4, b5 represent the bias of the hidden layer in the analytical model;

[0196] ReLU(·) represents the activation function;

[0197] From the true posterior probability distribution p(x k |z k,s ) Extract the distribution parameter μ′ k,s, (σ′ k,s ) 2 Generate image pixel distribution x k (s), where x k (s) = μ′ k,s =w5tanh(w4z k,s +b4)+b5, pixel distribution x k The standard deviation σ′ of (s) k,s is log(σ′ k,s ) 2 =ReLU[w5tanh(w4z k,s +b4)+b5];

[0198] S25: Based on image pixel distribution x k (s) Generate driving status image.

[0199] S3: Determine an adaptive driving scenario generation optimization objective function according to the constructed adaptive driving scenario generation model.

[0200] Determining the adaptive driving scenario generation optimization objective function in step S3 includes:

[0201] An adaptive driving scene generation optimization objective function is determined according to the constructed adaptive driving scene generation model, wherein the optimization goal of the adaptive driving scene generation optimization objective function is to make the approximate posterior probability distribution closer to the true posterior probability distribution and generate a more effective latent space encoding vector;

[0202] The determined adaptive driving scenario generation optimization objective function F(θ1, θ2) is:

[0203]

[0204] in:

[0205] represents the approximate posterior probability distribution q(z|x k ) and the true posterior probability distribution p(x k |z), where z represents the distribution of latent space encoding vectors;

[0206] θ1, θ2 represent the encoding model parameters and parsing model parameters respectively;

[0207] When the latent space encoding vector distribution z and the approximate posterior probability distribution q(z|x k ) are consistent, the KL divergence is minimized, and the KL divergence is simplified based on the output of the encoding model:

[0208]

[0209] in:

[0210] L represents the grayscale range, μ k (L), σ k (L) represents the Gaussian distribution parameter in L dimension, which is the output result of the coding model based on parameter θ1;

[0211] And simplify the true posterior probability distribution in the function to the L1 norm:

[0212]

[0213] The simplified adaptive driving scenario generation optimization objective function F′(θ1, θ2) is:

[0214]

[0215] in:

[0216] ||·|| denotes the L1 norm.

[0217] S4: Based on the collected driving state image set, the improved L-BFGS algorithm is used to optimize the adaptive driving scene generation optimization objective function to obtain the optimal model parameters.

[0218] In step S4, based on the collected driving state image set, an improved L-BFGS algorithm is used to optimize and solve the adaptive driving scene generation optimization objective function, including:

[0219] Based on the collected driving state image set, the improved L-BFGS algorithm is used to optimize the adaptive driving scene generation optimization objective function to obtain the optimal model parameters, wherein the model parameters to be optimized include the encoding model parameters θ1 and the analytical model parameters θ2, the encoding model parameters include the weight coefficients and biases of the hidden layers in the encoding model, and the analytical model parameters include the weight coefficients and biases of the hidden layers in the analytical model;

[0220] The optimization process of adaptive driving scenario generation optimization objective function is as follows:

[0221] S41: Initialize and generate n particles, each particle has a coordinate dimension of 10, corresponding to 10 model parameters to be optimized, where the initial coordinates loc(d0) and velocity v(d0) of any d-th particle are:

[0222]

[0223]

[0224] in:

[0225] represents the initial coordinates of the dth particle in the 10th dimension, The update speed of the coordinates corresponding to the dimension;

[0226] S42: Set the current iteration number of the optimization solution algorithm to t and the maximum iteration number to Max. Then the coordinate value of any d-th particle in the t-th iteration is loc(d t ), the corresponding speed is v(d t ), where the initial value of t is 1;

[0227] S43: At the beginning of each iteration, the coordinates of the n particles are respectively constructed as a coding model and a decoding model, and the driving state image set is input into the constructed coding model and decoding model to obtain the adaptive driving scene generation optimization objective function value F′(loc(d t )), where F′(loc(d t )) represents the optimization objective function value of the adaptive driving scenario generation based on the coordinate value of the d-th particle of the t-th iteration;

[0228] S44: Update the position coordinates of each particle using the L-BFGS algorithm:

[0229]

[0230]

[0231]

[0232] in:

[0233] I is the identity matrix;

[0234] a t-1 =loc(d t )-loc(d t-1 ), a0=d0;

[0235]

[0236] D0 is the identity matrix;

[0237] Indicates the calculation of gradient;

[0238] β1 represents the initial contraction coefficient, β1 = 1;

[0239] β2 represents the final shrinkage coefficient, β2 = 0.5;

[0240] S45: Update the velocity of each particle:

[0241]

[0242] in:

[0243] α represents the learning rate, which is set to 0.2. It represents the speed of the particle with the smallest gradient change rate at the t-1th iteration after the position coordinates are updated, where the gradient change rate of the dth particle after the position coordinates are updated is

[0244] S46: Calculate the gradient change rate of each particle after the tth iteration. If there is a particle whose gradient change rate is less than the preset threshold ∈, then end the update iteration of the particle position and velocity, set t=t+1, and return to step S43;

[0245] S47: When all particles have finished iteration or reached the maximum number of iterations Max, the adaptive driving scene generation optimization objective function value of each particle is calculated, and the particle position with the smallest adaptive driving scene generation optimization objective function value is selected as the optimal model parameter obtained by solving.

[0246] S5: construct an optimal adaptive driving scene generation model according to the optimal model parameters obtained by solving, and input images in the driving state image set into the model to generate a large number of driving state images.

[0247] In step S5, the optimal adaptive driving scene generation model is constructed according to the optimal model parameters obtained by solving, including:

[0248] A coding model and a parsing model are constructed according to the solved weight coefficients and the bias, and the constructed coding model and the parsing model are used as the constructed optimal adaptive driving scene generation model.

[0249] In step S5, the images in the driving state image set are input into the model to generate a large number of driving state images, including:

[0250] The images in the driving state image set are input into the encoding model of the optimal adaptive driving scene generation model to obtain the Gaussian distribution parameters of each image, and S different noises are added to the Gaussian distribution parameters of each image in turn to generate S latent space encoding vectors, and the S latent space encoding vectors are input into the parsing model in turn to obtain S generated images of the input image.

[0251] S6: Using the collected and generated driving status images as a training set, training the driver status analysis neural network, and using the trained driver status analysis neural network to perform driver status analysis and judgment.

[0252] In step S6, the collected and generated driving state images are used as a training set to train the driver state analysis neural network, including:

[0253] Constructing a driver state analysis neural network, wherein the driver state analysis neural network comprises an encoding layer, a facial feature detection layer and a state analysis layer. The input of the encoding layer is a driving state image. The driving state image is convolutionally encoded using a convolution module, and the encoding result is input into the facial feature detection layer. The structure of the facial feature detection layer is a YOLOv3 model. The facial feature regions of the driver are marked using the facial feature detection layer, and the mouth opening and eye opening are calculated in the state analysis layer to determine the driver's fatigue state. When the eye opening in the driver's driving state image sequence is mostly lower than a preset threshold value ω1, and the mouth opening is mostly higher than a preset threshold value ω2, it indicates that the driver is in a fatigue state.

[0254] The collected and generated driving state images are used as training set data to train the driver state analysis neural network. The training objective function of the driver state analysis neural network is:

[0255]

[0256] in:

[0257] IOU c (m) represents the intersection-and-union ratio between the mth facial feature region obtained by marking the encoding result of the driving state image c in the training set data by the facial feature detection layer and the real facial feature region;

[0258] The driver state neural network is trained using the gradient descent method, and the trained driver state analysis neural network is used to analyze and judge the driver state.

[0259] Embodiment 2:

[0260] like Figure 2 , is a schematic diagram of the structure of an electronic device for implementing a method for analyzing and judging a driver's state based on a deep learning neural network provided by an embodiment of the present invention.

[0261] The electronic device 1 may include a processor 10 , a memory 11 , a communication interface 13 and a bus, and may also include a computer program stored in the memory 11 and executable on the processor 10 , such as a program 12 .

[0262] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. The memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, such as a mobile hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Further, the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 may be used not only to store application software and various types of data installed in the electronic device 1, such as the code of the program 12, etc., but also to temporarily store data that has been output or is to be output.

[0263] The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules stored in the memory 11 (such as the program 12 for analyzing and judging the driver's state), and calls data stored in the memory 11, so as to execute various functions of the electronic device 1 and process data.

[0264] The communication interface 13 may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices, and to achieve connection and communication between internal components of the electronic device.

[0265] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0266] Figure 2 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 2 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0267] For example, although not shown, the electronic device 1 may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The electronic device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0268] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0269] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0270] The program 12 stored in the memory 11 in the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve:

[0271] Collecting and preprocessing the driver's driving state images to obtain a standardized driving state image set;

[0272] Build an adaptive driving scenario generation model;

[0273] Determining an adaptive driving scenario generation optimization objective function according to the constructed adaptive driving scenario generation model;

[0274] Based on the collected driving state image set, the improved L-BFGS algorithm is used to optimize the adaptive driving scene generation optimization objective function to obtain the optimal model parameters;

[0275] An optimal adaptive driving scene generation model is constructed according to the optimal model parameters obtained by solving the problem, and images in the driving state image set are input into the model to generate a large number of driving state images;

[0276] The collected and generated driving status images are used as training sets to train the driver status analysis neural network, and the trained driver status analysis neural network is used to perform driver status analysis and judgment.

[0277] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to Figure 1 to Figure 2 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0278] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof in this article are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0279] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0280] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A driver status analysis and judgment method based on deep learning neural network, characterized in that: The method comprises: S1: Collecting and preprocessing the driver's driving state images to obtain a standardized driving state image set; S2: Build an adaptive driving scenario generation model; including: Constructing an adaptive driving scene generation model, wherein the adaptive driving scene generation model includes an encoding model and a parsing model, wherein the encoding model takes a normalized driving state image as input and a corresponding latent space encoding vector as output, and the parsing model takes a latent space encoding vector as input and a generated multi-scenario driving state image as output; The process of generating multi-scenario driving state images based on the adaptive driving scene generation model is as follows: S21: Calculate the pixel distribution x of the input k-th normalized driving state image k : x k ={p k,L |L∈[0,255]} in: p k,L represents the probability distribution of the gray level L in the k-th normalized driving state image, count(L, k) represents the number of pixels with gray value L in the k-th normalized driving state image, k∈[1, K]; S22: Distribute pixels x k Input to the encoding model: μ k =w2[tanh(w1x k +b1)]+b2 in: q(z|x k ) represents the approximate posterior probability distribution, z represents the latent space encoding vector distribution; It means that given any x k , z all obey the multivariate Gaussian distribution independent of each dimension I represents the identity matrix, represents z|x k The parameters of the Gaussian distribution that it obeys; w1, w2, w3 represent the weight coefficients of the hidden layer in the encoding model, b1, b2, b3 represent the bias of the hidden layer in the encoding model; S23: Coding model based on the solved Gaussian distribution parameter μ k , Generate x k The latent space encoding vector of : z k,s =μ k +σk⊙ε s ,he s ~N(0,1) in: z k,s represents the generated noise ε added s x k The latent space encoding vector, ε s ~N(0,1) represents the noise ε s It conforms to the Gaussian distribution, s∈[1,S], S represents the number of types of noise added; ⊙ represents element-by-element multiplication; S24: Input the latent space encoding vector into the parsing model to obtain the true posterior probability distribution p(x k |z k,s ): log p(x k |z k,s )=x k log z′ k ,s+(1-x k )log(1-z′ k,s ) z′ k,s =ReLU[w5tanh(w4z k,s +b4)+b5] in: w4, w5 represent the weight coefficients of the hidden layer in the analytical model, b4, b5 represent the bias of the hidden layer in the analytical model; ReLU(·) represents the activation function; From the true posterior probability distribution p(x k |z k,s ) Extract the distribution parameter μ′ k,s’ (σ′k,s) 2 Generate image pixel distribution x k (s), where x k (s) = μ′ k,s =w5tanh(w4z k,s +b4)+b5, pixel distribution x k The standard deviation σ′ of (s) k,s is log(σ′ k,s ) 2 =ReLU[w5tanh(w4z k,s +b4)+b5]; S25: Based on image pixel distribution x k (s) generating a driving status image; S3: determining an adaptive driving scenario generation optimization objective function according to the constructed adaptive driving scenario generation model; S4: Based on the collected driving state image set, the adaptive driving scene generation optimization objective function is optimized and solved using the improved L-BFGS algorithm to obtain the optimal model parameters; including: Based on the collected driving state image set, the improved L-BFGS algorithm is used to optimize the adaptive driving scene generation optimization objective function to obtain the optimal model parameters, wherein the model parameters to be optimized include the encoding model parameters θ1 and the analytical model parameters θ2, the encoding model parameters include the weight coefficients and biases of the hidden layers in the encoding model, and the analytical model parameters include the weight coefficients and biases of the hidden layers in the analytical model; The optimization process of adaptive driving scenario generation optimization objective function is as follows: S41: Initialize and generate n particles, each particle has a coordinate dimension of 10, corresponding to 10 model parameters to be optimized, where the initial coordinates loc(d0) and velocity v(d0) of any d-th particle are: in: represents the initial coordinates of the dth particle in the 10th dimension, The update speed of the coordinates corresponding to the dimension; S42: Set the current iteration number of the optimization solution algorithm to t and the maximum iteration number to Max. Then the coordinate value of any d-th particle in the t-th iteration is loc(d t ), the corresponding speed is v(d t ), where the initial value of t is 1; S43: At the beginning of each iteration, the coordinates of the n particles are respectively constructed as a coding model and a decoding model, and the driving state image set is input into the constructed coding model and decoding model to obtain the adaptive driving scene generation optimization objective function value F′(loc(d t )), where F′(loc(d t )) represents the optimization objective function value of the adaptive driving scenario generation based on the coordinate value of the d-th particle of the t-th iteration; S44: Update the position coordinates of each particle using the L-BFGS algorithm: in: I is the identity matrix; to t-1 =loc(d t )-loc(d t-1 ),a0=d0; D0 is the identity matrix; Indicates the calculation of gradient; β1 represents the initial contraction coefficient, β1 = 1; β2 represents the final shrinkage coefficient, β2 = 0.5; S45: Update the velocity of each particle: in: α represents the learning rate, which is set to 0.

2. It represents the speed of the particle with the smallest gradient change rate at the t-1th iteration after the position coordinates are updated, where the gradient change rate of the dth particle after the position coordinates are updated is S46: Calculate the gradient change rate of each particle after the tth iteration. If there is a particle whose gradient change rate is less than the preset threshold ∈, then end the update iteration of the particle position and velocity, set t=t+1, and return to step S43; S47: When all particles have finished iteration or reached the maximum number of iterations Max, the adaptive driving scene generation optimization objective function value of each particle is calculated, and the particle position with the smallest adaptive driving scene generation optimization objective function value is selected as the optimal model parameter obtained by solving; S5: constructing an optimal adaptive driving scene generation model according to the optimal model parameters obtained by solving, and inputting images in the driving state image set into the model to generate a large number of driving state images; S6: Using the collected and generated driving status images as a training set, training the driver status analysis neural network, and using the trained driver status analysis neural network to perform driver status analysis and judgment.

2. A driver status analysis and judgment method based on deep learning neural network as claimed in claim 1, characterized in that: The step S1 includes collecting the driver's driving state image and preprocessing it, including: The driver's driving state image is collected during the driver's driving process, wherein the driver's driving state image is an image of the driver driving the vehicle in different scenes and at different angles; The collected driver driving state images are preprocessed to obtain driver driving state images with the same image specifications and in grayscale, and all preprocessed driver driving state images constitute a driving state image set, wherein the preprocessing process is: S11: finding the maximum value of the three RGB color components of each pixel in the driver's driving state image, and setting the maximum value as the grayscale value of the pixel to obtain a grayscale image of the driver's driving state image; S12: performing image normalization processing on the grayscale image of the driver's driving state with any size of M×N pixels, wherein the formula for image normalization processing is: in: (i, j) represents the pixel in the i-th row and j-th column of the grayscale image of the driver's driving state image with a size of M×N pixels, M represents the number of row pixels of the driver's driving state image, N represents the number of column pixels of the driver's driving state image, and (i′, j′) represents the pixel after normalization processing, that is, the pixel in the i-th row and j-th column is mapped to the i′th row and j′th column; M′×N′ pixels represent the grayscale image specifications of the driver’s driving status image after normalization; S13: Calculate the grayscale value average of the adjacent pixels of the empty pixel, and use the grayscale value average as the grayscale value of the empty pixel, wherein the empty pixel represents a pixel without a grayscale value in the grayscale image after normalization processing; S14: stretching the grayscale value of each pixel in the normalized grayscale image: in: Gray(i′, j′) represents the gray value of pixel (i′, j′) in the grayscale image after normalization; MIN Gray It is the minimum grayscale value in the grayscale image after normalization; MAX Gray It is the maximum grayscale value in the grayscale image after normalization; g(i′, j′) is the grayscale value of pixel (i′, j′) after stretching.

3. A driver status analysis and judgment method based on deep learning neural network as claimed in claim 2, characterized in that: Determining the adaptive driving scenario generation optimization objective function in step S3 includes: An adaptive driving scene generation optimization objective function is determined according to the constructed adaptive driving scene generation model, wherein the optimization goal of the adaptive driving scene generation optimization objective function is to make the approximate posterior probability distribution closer to the true posterior probability distribution and generate a more effective latent space encoding vector; The determined adaptive driving scenario generation optimization objective function F(θ1, θ2) is: in: p(x k |z)) represents the approximate posterior probability distribution q(z|x k ) and the true posterior probability distribution p(x k |z), where z represents the distribution of latent space encoding vectors; θ1, θ2 represent the encoding model parameters and parsing model parameters respectively; When the latent space encoding vector distribution z and the approximate posterior probability distribution q(z|x k ) are consistent, the KL divergence is minimized, and the KL divergence is simplified based on the output of the encoding model: in: L represents the grayscale range, μ k (L), σ k (L) represents the Gaussian distribution parameter in L dimension, which is the output result of the coding model based on parameter θ1; And simplify the true posterior probability distribution in the function to the L1 norm: The simplified adaptive driving scenario generation optimization objective function F′(θ1, θ2) is: in: ||·|| denotes the L1 norm.

4. A driver status analysis and judgment method based on deep learning neural network as claimed in claim 3, characterized in that: In step S5, the optimal adaptive driving scene generation model is constructed according to the optimal model parameters obtained by solving, including: A coding model and a parsing model are constructed according to the solved weight coefficients and the bias, and the constructed coding model and the parsing model are used as the constructed optimal adaptive driving scene generation model.

5. The driver status analysis and judgment method based on deep learning neural network according to claim 1, characterized in that: In step S5, the images in the driving state image set are input into the model to generate a large number of driving state images, including: The images in the driving state image set are input into the encoding model of the optimal adaptive driving scene generation model to obtain the Gaussian distribution parameters of each image, and S different noises are added to the Gaussian distribution parameters of each image in turn to generate S latent space encoding vectors, and the S latent space encoding vectors are input into the parsing model in turn to obtain S generated images of the input image.

6. A driver status analysis and judgment method based on deep learning neural network as claimed in claim 5, characterized in that: In step S6, the collected and generated driving state images are used as a training set to train the driver state analysis neural network, including: Constructing a driver state analysis neural network, wherein the driver state analysis neural network comprises an encoding layer, a facial feature detection layer and a state analysis layer. The input of the encoding layer is a driving state image. The driving state image is convolutionally encoded using a convolution module, and the encoding result is input into the facial feature detection layer. The structure of the facial feature detection layer is a YOLOv3 model. The facial feature regions of the driver are marked using the facial feature detection layer, and the mouth opening and eye opening are calculated in the state analysis layer to determine the driver's fatigue state. When the eye opening in the driver's driving state image sequence is mostly lower than a preset threshold value ω1, and the mouth opening is mostly higher than a preset threshold value ω2, it indicates that the driver is in a fatigue state. The collected and generated driving state images are used as training set data to train the driver state analysis neural network. The training objective function of the driver state analysis neural network is: in: IOU c (m) represents the intersection-and-union ratio between the mth facial feature region obtained by marking the encoding result of the driving state image c in the training set data by the facial feature detection layer and the real facial feature region; The driver state neural network is trained using the gradient descent method, and the trained driver state analysis neural network is used to analyze and judge the driver state.