Driver state estimation method, device, equipment, storage medium and program product

Abstract

The invention provides a driver state estimation method and device, equipment, a storage medium and a program product, and relates to the technical field of vehicles. According to the driver state estimation method, modeling of dynamic evolution characteristics of a driver state is realized by constructing a state transition matrix for describing a time sequence evolution rule of a hidden state, namely a driver real state, and a transmission matrix for describing probability distribution of a state observation result in different hidden states is constructed. According to the method, correlation analysis of hidden state and historical state observation results is realized, state enhancement processing is further performed on a state observation sequence corresponding to the current moment through a state enhancement model in combination with a target state transition matrix and an emission matrix, correction of the state observation result at the current moment is realized, and the accuracy of driver state estimation is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicles, in particular to a driver state estimation method and device, equipment, storage medium and program product. BACKGROUND

[0002] Driver fatigue is one of the important factors leading to traffic accidents. According to reports, 60% of fatal accidents of heavy trucks are caused by driver fatigue. However, many drivers may not realize that they are in a state of fatigue and continue to drive, which undoubtedly increases the risk of accidents. Therefore, it is of great significance for a vehicle to issue an alarm in time when a driver is in a state of fatigue. At present, many vehicles are equipped with a driver monitoring system (DMS) which can be used to monitor whether a driver is fatigued.

[0003] In related technologies, the DMS usually estimates the driver state based on the driver's face image. Specifically, a camera installed in the cockpit captures the driver's face image in real time, and the DMS obtains the driver's face image and extracts the driver's eye aspect ratio, mouth opening degree and head posture and other facial key features from the driver's face image based on an image processing algorithm, and then estimates the driver state based on these facial key features.

[0004] However, the above-mentioned DMS for estimating the driver state based on the driver's face image usually estimates only based on the current driver face image frame, lacks dynamic modeling of time series data, and is difficult to distinguish between transient abnormalities (such as short-term eye closure) and true fatigue states (such as sustained attention decline), thus having the problem of low estimation accuracy. SUMMARY

[0005] The present application provides a driver state estimation method, device, equipment, storage medium and program product to solve the problem of low estimation accuracy of the DMS for estimating the driver state based on the driver's face image in related technologies.

[0006] In a first aspect, the present application provides a driver state estimation method, comprising: obtaining a state observation sequence corresponding to a current time, the state observation sequence comprising at least two state observation results; inputting the state observation sequence into a state enhancement model for state enhancement processing, combining an emission matrix and a target state transition matrix to obtain a target hidden state probability distribution corresponding to the current time, the target state transition matrix being used to describe the target time sequence evolution rule of the hidden state at the current time, and the emission matrix being used to describe the probability distribution of the state observation result under different hidden states; and estimating a target driver state corresponding to the current time according to the target hidden state probability distribution.

[0007] The application realizes modeling of dynamic evolution characteristics of the driver state by constructing a state transition matrix for describing the time evolution law of the hidden state, i.e., the real state of the driver, and realizes correlation analysis of the hidden state and the historical state observation by constructing an emission matrix for describing the probability distribution of the state observation result under different hidden states. Further, the application realizes correction of the state observation result at the current moment by state enhancement processing of the state observation sequence at the current moment in combination with the emission matrix and the target state transition matrix through a state enhancement model, thereby improving the accuracy of the driver state estimation.

[0008] In a possible implementation, the state observation sequence is input into the state enhancement model for state enhancement processing, and the target hidden state probability distribution corresponding to the current moment is obtained in combination with the emission matrix and the target state transition matrix, including: inputting the state observation sequence into the state enhancement model, combining the emission matrix, the target state transition matrix and the preset hidden state set, calculating the target forward probability and the target backward probability corresponding to the current moment; and calculating the target hidden state probability distribution according to the target forward probability and the target backward probability.

[0009] The application improves the capturing ability of the progressive change of the fatigue state, realizes joint modeling of the state transition characteristics of the historical state observation result, improves the accuracy of the driver state estimation, and can optimize the robustness of the driver state derivation by inputting the state observation sequence into the state enhancement model, combining the emission matrix, the target state transition matrix and the preset hidden state set, calculating the target forward probability and the target backward probability corresponding to the current moment, and calculating the target hidden state probability distribution according to the target forward probability and the target backward probability.

[0010] In a possible implementation, the state observation sequence is input into the state enhancement model, the target forward probability is calculated by combining the emission matrix, the target state transition matrix and the preset hidden state set using a forward algorithm, and the target backward probability is calculated by combining the emission matrix, the target state transition matrix and the preset hidden state set using a backward algorithm.

[0011] The application improves the capturing ability of the progressive change of the fatigue state and realizes joint modeling of the state transition characteristics of the historical state observation result by calculating the target forward probability and the target backward probability based on the forward algorithm and the backward algorithm respectively.

[0012] In a possible implementation, before the target hidden state probability distribution corresponding to the current moment is obtained by combining the emission matrix and the target state transition matrix, the driver state estimation method further includes: determining whether the state probability representation coefficient corresponding to the current moment converges; when the state probability representation coefficient does not converge, the state enhancement model performs state enhancement processing based on the state observation sequence, combines the emission matrix and the initial state transition matrix at the current moment, calculates the hidden state probability distribution and the state transition probability distribution at any moment n, where n is the time index of the state observation sequence; and according to the hidden state probability distribution and the state transition probability distribution at any moment n, the initial state transition matrix is calculated and updated until the state probability representation coefficient converges, to obtain the target state transition matrix.

[0013] The application determines whether the state probability representation coefficient corresponding to the current moment converges, and when the state probability representation coefficient does not converge, the state enhancement model performs state enhancement processing based on the state observation sequence, combines the emission matrix and the initial state transition matrix at the current moment, calculates the hidden state probability distribution and the state transition probability distribution at any moment n, further calculates and updates the initial state transition matrix according to the hidden state probability distribution and the state transition probability distribution at any moment n, until the state probability representation coefficient converges, to obtain the target state transition matrix, improves the adaptability of the state transition matrix to the time sequence evolution rule of the real state of the driver, enhances the dynamic correlation of the driver state estimation, and further improves the estimation accuracy of the driver state.

[0014] In a possible implementation, the state enhancement model performs state enhancement processing based on the state observation sequence, combines the emission matrix and the initial state transition matrix at the current moment, and calculates the hidden state probability distribution and the state transition probability distribution at any moment n, including: calculating the forward probability and the first backward probability at any moment n according to the state observation sequence, the initial state transition matrix, the emission matrix and the preset hidden state set; calculating the hidden state probability distribution at any moment n according to the forward probability and the first backward probability; and calculating the state transition probability distribution at any moment n according to the forward probability, the initial state transition matrix, the emission matrix, the second backward probability at moment n+1 and the state probability representation coefficient.

[0015] In a possible implementation, the state observation sequence at the current moment is obtained, including: obtaining the driver face image corresponding to each moment in a target period, the target period being a period corresponding to the starting moment and the current moment; estimating the state observation result corresponding to each moment according to the driver face image corresponding to each moment; and arranging all the state observation results corresponding to the target period in time sequence to obtain the state observation sequence.

[0016] In a second aspect, the application provides a driver state estimation device, including:

[0017] an acquisition module configured to acquire a state observation sequence corresponding to a current time, the state observation sequence including at least two state observation results;

[0018] an enhancement processing module configured to input the state observation sequence into a state enhancement model to perform state enhancement processing, and obtain a target hidden state probability distribution corresponding to the current time by combining an emission matrix and a target state transition matrix, the target state transition matrix being used to describe a target time sequence evolution rule of a hidden state at the current time, and the emission matrix being used to describe a probability distribution of a state observation result under different hidden states;

[0019] an estimation module configured to estimate a target driver state corresponding to the current time according to the target hidden state probability distribution.

[0020] In a possible implementation, the enhancement processing module is specifically configured to input the state observation sequence into the state enhancement model, combine the emission matrix, the target state transition matrix, and a preset hidden state set, calculate a target forward probability and a target backward probability corresponding to the current time, and calculate the target hidden state probability distribution according to the target forward probability and the target backward probability.

[0021] In a possible implementation, the enhancement processing module is further configured to input the state observation sequence into the state enhancement model, combine the emission matrix, the target state transition matrix, and the preset hidden state set, calculate the target forward probability by using a forward algorithm, and calculate the target backward probability by using a backward algorithm.

[0022] In a possible implementation, before the target hidden state probability distribution corresponding to the current time is obtained by combining the emission matrix and the target state transition matrix, the driver state estimation apparatus further includes an updating module (not shown in the figure), which is specifically configured to determine whether a state probability representation coefficient corresponding to the current time converges, perform state enhancement processing on the state observation sequence by the state enhancement model when the state probability representation coefficient does not converge, combine the emission matrix and an initial state transition matrix at the current time to calculate a hidden state probability distribution and a state transition probability distribution at any time n, n being a time index of the state observation sequence, and calculate and update the initial state transition matrix according to the hidden state probability distribution and the state transition probability distribution at any time n until the state probability representation coefficient converges, to obtain the target state transition matrix.

[0023] In a possible implementation, the updating module is further configured to: calculate a forward probability and a first backward probability of any time n according to the state observation sequence, the initial state transition matrix, the emission matrix and the preset hidden state set; calculate a hidden state probability distribution of any time n according to the forward probability and the first backward probability; and calculate a state transition probability distribution of any time n according to the forward probability, the initial state transition matrix, the emission matrix, a second backward probability of time n+1 and a state probability representation coefficient.

[0024] In a possible implementation, the obtaining module is specifically configured to: obtain a driver face image corresponding to each time in a target period, the target period being a period corresponding to a time between a starting time and a current time; estimate a state observation result corresponding to each time according to the driver face image corresponding to each time; and arrange all the state observation results corresponding to the target period in time sequence to obtain the state observation sequence.

[0025] In a third aspect, the present application provides a vehicle, comprising a vehicle body and a driver state estimation system, the driver state estimation system being configured to implement the driver state estimation method provided in the first aspect.

[0026] In a fourth aspect, the present application provides an electronic device, comprising: a processor and a memory connected with the processor in communication; the memory stores computer execution instructions; and the processor executes the computer execution instructions stored in the memory to implement the driver state estimation method provided in the first aspect.

[0027] In a fifth aspect, the present application provides a computer readable storage medium, the computer readable storage medium storing computer execution instructions, the computer execution instructions being executed by a processor to implement the driver state estimation method provided in the first aspect.

[0028] In a fifth aspect, the present application provides a computer program product, comprising: a computer program, the computer program being executed by a processor to implement the driver state estimation method provided in the first aspect.

[0029] The driver state estimation method, apparatus, device, storage medium, and program product provided in this application acquire a state observation sequence corresponding to the current moment, which includes at least two state observation results. The state observation sequence is input into a state enhancement model for state enhancement processing. By combining the emission matrix and the target state transition matrix, the probability distribution of the target hidden state corresponding to the current moment is obtained. This target state transition matrix describes the temporal evolution law of the hidden state at the current moment, and the emission matrix describes the probability distribution of the state observation results under different hidden states. Based on the probability distribution of the target hidden state, the target driver state corresponding to the current moment is estimated. This application achieves modeling of the dynamic evolution characteristics of the driver state by constructing a state transition matrix to describe the temporal evolution law of the hidden state, i.e., the driver's true state. It also achieves correlation analysis between the hidden state and historical state observation results by constructing an emission matrix to describe the probability distribution of the state observation results under different hidden states. Furthermore, through the state enhancement model, combined with the emission matrix and the target state transition matrix, state enhancement processing is performed on the state observation sequence corresponding to the current moment to correct the state observation results at the current moment and improve the accuracy of driver state estimation. Attached Figure Description

[0030] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0031] Figure 1 A diagram showing the comparison between the driver's status output by DMS and the actual driver's status;

[0032] Figure 2 A schematic diagram of the driver's status output by DMS;

[0033] Figure 3 This is an architecture diagram of the driver state estimation system provided in the embodiments of this application;

[0034] Figure 4 A flowchart illustrating the driver state estimation method provided in this application embodiment. Figure 1 ;

[0035] Figure 5 A schematic diagram of a hidden Markov process provided for an embodiment of this application;

[0036] Figure 6 A flowchart illustrating the driver state estimation method provided in this application embodiment. Figure 2 ;

[0037] Figure 7 A flowchart illustrating the driver state estimation method provided in this application embodiment. Figure 3 ;

[0038] Figure 8 A comparison diagram of driver states obtained by different estimation manners provided by an embodiment of the present application;

[0039] Figure 9 A structure diagram of a driver state estimation device provided by an embodiment of the present application;

[0040] Figure 10 A structure diagram of an electronic device provided by an embodiment of the present application.

[0041] The specific embodiments of the present application have been shown through the above-described drawings, and will be described in more detail hereinafter. These drawings and the written description are not intended to restrict the scope of the present application by any means, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0042] The exemplary embodiments will be described in detail herein with reference to the accompanying drawings. Unless otherwise indicated, the same numbers on different drawings represent the same or similar elements. The following detailed description does not, therefore, limit the scope of the application, but the concept of the present application is applicable to other implementations and applications.

[0043] In the related art, the estimation of the driver state mainly depends on an estimation manner based on vehicle behavior and an estimation manner based on a driver facial image. In the estimation manner based on vehicle behavior, it is assumed that a driver in a fatigue state will have difficulty in controlling a steering wheel, and a DMS estimates the driver state by monitoring the steering wheel angle of the vehicle. However, in this estimation manner, different drivers can have different driving behaviors, and the driving behavior of the driver can be affected by road conditions, which can cause many false reports of the driver state and result in low estimation accuracy.

[0044] In the estimation manner based on the driver facial image, a camera installed in the cockpit is used to collect a driver facial image in real time, and the DMS extracts the eye aspect ratio, mouth opening degree, and head posture of the driver from the driver facial image based on an image processing algorithm after obtaining the driver facial image. Then, the driver state is estimated based on these facial key features. However, in this estimation manner, the estimation is usually based on only the current driver facial image frame, and there is a lack of dynamic modeling of time series data, which makes it difficult to distinguish between transient abnormalities (such as temporary eye closure) and real fatigue states (such as continuous attention decline), and there is a problem of low estimation accuracy.

[0045] Currently, when using a Driver Monitoring System (DMS) to monitor driver fatigue, regardless of whether the estimation method is based on vehicle behavior or driver facial images, the DMS outputs the driver's fatigue status at a specific frequency, such as once every 10 seconds. However, because the DMS does not consider the evolution of the driver's status over time or its correlation with historical driver states when estimating the driver's status, the estimated driver status is inaccurate, resulting in false alarms (the driver is not fatigued, but the DMS outputs a fatigued state) and false negatives (the driver is actually fatigued, but the DMS outputs a non-fatigue state).

[0046] Figure 1 This diagram illustrates the comparison between the driver's state output by the DMS and the actual driver's state. In this example, the accuracy and precision of the DMS are both set to 95%. Figure 1 As shown, the horizontal axis represents time, in minutes, and the vertical axis represents the driver's state. From Figure 1 As can be seen, although the accuracy and precision of the DMS are both 95%, there are still a large number of false alarms (also known as false positives) and false negatives (also known as false negatives) in the driver state output by the DMS throughout the entire driver state simulation process. However, a high false alarm rate will reduce the driver's trust in the driver state output by the DMS, and may even distract the driver.

[0047] In related technologies, traditional algorithms reduce false alarms by setting higher driver fatigue detection thresholds in the DMS (Driver Monitoring System). However, raising the driver fatigue detection threshold leads to more missed alarms, reduces system sensitivity, and may even result in the inability to detect driver fatigue. Therefore, traditional algorithms struggle to improve the monitoring accuracy of the DMS.

[0048] In related technologies, DMS classifies driver fatigue levels into different levels, such as four levels from level 0 to level 3. Level 0 indicates that the driver is not fatigued, while level 3 indicates that the driver is severely fatigued. Figure 2 This is a diagram illustrating the driver's status output by the DMS. (Example) Figure 2 As shown, the driver fatigue level frequently fluctuates between (or vice versa) level 0 and level 2, which is unreasonable in real-world scenarios. This inconsistency arises because the DMS estimates the driver's state only based on information at the current moment, without considering historical fatigue levels, resulting in low estimation accuracy.

[0049] Based on the technical problems existing in the related art, the embodiments of the present application realize modeling of the dynamic evolution characteristics of the driver state by constructing a state transition matrix for describing the time evolution law of the hidden state, i.e., the real state of the driver, and realize correlation analysis of the hidden state and the historical state observation by constructing an emission matrix for describing the probability distribution of the state observation result under different hidden states. Further, the embodiments of the present application realize correction of the state observation result at the current moment by performing state enhancement processing on the state observation sequence corresponding to the current moment by means of the state enhancement model of the state transition matrix and the emission matrix, thereby improving the accuracy of the driver state estimation.

[0050] Firstly, the application scenarios of the embodiments of the present application are described below.

[0051] The driver state estimation method provided by the embodiments of the present application is suitable for real-time monitoring of the driver state in a vehicle driving environment. Specifically, it is suitable for various vehicle types such as heavy trucks, passenger cars, and public transportation vehicles equipped with DMS, for detecting whether the driver is in a fatigue state or a drowsy state, and timely issuing a warning to avoid traffic accidents.

[0052] Figure 3 The architecture diagram of the driver state estimation system provided by the embodiments of the present application is shown in FIG. 1. As shown in FIG. 1, the driver state estimation system provided by the embodiments of the present application includes a DMS and a state enhancement module, and the state enhancement module includes a processing module 31 and a processing module 32. Figure 3

[0053] For example, a state enhancement model is provided in the state enhancement module, and the state enhancement model can be a Hidden Markov Model (HMM). The model parameters of the state enhancement model include a state transition matrix and an emission matrix.

[0054] As shown in FIG. 2, the state enhancement module is configured to receive the state observation sequence and the state transition matrix and the emission matrix, and perform state enhancement processing on the state observation sequence by means of the state enhancement model, thereby obtaining the state observation result at the current moment. Figure 3 ​As shown, in the driver state estimation system, in the processing module 31 of the state enhancement module, when the state probability representation coefficient corresponding to the current time has not converged, the forward probability, backward probability, and hidden state probability distribution at any time n are calculated based on the state observation sequence containing at least two state observation results and the initial state transition matrix corresponding to the current time output by the DMS. The state transition probability distribution at any time n is further calculated, and then the initial state transition matrix is ​​calculated and updated based on the hidden state probability distribution and the state transition probability distribution at any time n until the state probability representation coefficient converges, thus obtaining the target state transition matrix corresponding to the current time. If the state probability representation coefficient has not converged, the above calculation and update process is repeated until the state probability representation coefficient converges. In the processing module 32 of the state enhancement module, the forward probability and backward probability of the target corresponding to the current time are calculated based on the target state transition matrix and the state observation sequence. The hidden state probability distribution of the target corresponding to the current time is then calculated, and the target driver state is estimated based on the hidden state probability distribution. This corrects the state observation results output by the DMS, reduces the false alarm rate of the driver state, and improves the accuracy of driver state estimation.

[0055] Understandable, Figure 3 The processing module 31 shown can be the update module (not shown) in the driver state estimation device of this application embodiment, and is used to calculate and update the initial state transition matrix. Figure 3 The processing module 32 shown can be the enhanced processing module and estimation module in the driver state estimation device of this application embodiment (e.g., Figure 9 As shown in the figure, it is used to input the state observation sequence into the state enhancement model for state enhancement processing. By combining the emission matrix and the target state transition matrix, the probability distribution of the target hidden state at the current moment is obtained, and the target driver state at the current moment is estimated based on the probability distribution of the target hidden state.

[0056] It is understood that the driver state estimation system provided in this application embodiment can be implemented by deploying a state enhancement module on the DMS without modifying the DMS.

[0057] like Figure 3 As shown, the driver state output by the driver state estimation system can be sent to downstream systems such as the fatigue warning system configured in the vehicle. When the estimated driver state is fatigued, the fatigue warning system will issue a warning to the driver, thereby improving driving safety.

[0058] The following is based on Figure 3The driver state estimation system shown in the middle is the execution subject, and the specific implementation of the driver state estimation method provided in the embodiments of the present application is described in detail.

[0059] Figure 4 The flowchart of the driver state estimation method provided in the embodiments of the present application Figure 1 As shown in the figure Figure 4 The specific implementation of the driver state estimation method can include the following steps:

[0060] S401, obtaining a state observation sequence corresponding to the current time, the state observation sequence including at least two state observation results.

[0061] For example, the state observation sequence can be a time sequence of state observation results from the start time to the current time output by the DMS, such as "awake-fatigue-awake-awake-fatigue". Wherein, the state observation result corresponding to each time point can be awake or fatigue.

[0062] For example, the time point can be determined according to the frequency of the DMS output state observation result, such as when the frequency of the DMS output state observation result is once every 10s, the corresponding time point can be 10s, 20s, 30s, and so on.

[0063] For example, when the current time is the 30th second, the state observation sequence can be a time sequence composed of the state observation results output by the DMS at the 10th second, the 20th second and the 30th second in chronological order.

[0064] S402, inputting the state observation sequence into the state enhancement model for state enhancement processing, combining the emission matrix and the target state transition matrix to obtain the target hidden state probability distribution corresponding to the current time, the target state transition matrix being used to describe the target time sequence evolution law of the hidden state at the current time, and the emission matrix being used to describe the probability distribution of the state observation result under different hidden states.

[0065] For example, the state enhancement model can be HMM.

[0066] It can be understood that in the driver fatigue state monitoring, the real state of the driver can be two, namely awake and fatigue. The real state of the driver changes over time with a certain probability, based on which, the embodiments of the present application define the state transition matrix in the state enhancement model to describe the time sequence evolution law of the hidden state of the driver, i.e. the real state of the driver changes over time.

[0067] For example, the state transition matrix can be represented by the following formula:

[0068]

[0069] wherein, A represents a state transition matrix, i and j represent the hidden state of the driver, i.e., the real state of the driver, represents an element of the state transition matrix A, represents the probability that the hidden state of the driver at the nth moment is sober, represents the probability that the hidden state of the driver at the (n-1)th moment is sober, represents the probability that the hidden state of the driver at the nth moment is fatigue, represents the probability that the hidden state of the driver at the (n-1)th moment is fatigue, represents the probability that the hidden state of the driver at the nth moment is sober and the hidden state of the driver at the (n-1)th moment is sober, represents the probability that the hidden state of the driver at the nth moment is fatigue and the hidden state of the driver at the (n-1)th moment is sober, represents the probability that the hidden state of the driver at the nth moment is sober and the hidden state of the driver at the (n-1)th moment is fatigue, represents the probability that the hidden state of the driver at the nth moment is fatigue and the hidden state of the driver at the (n-1)th moment is fatigue.

[0070] Exemplarily, the state transition matrix can be a matrix of .

[0071] It can be understood that in the state enhancement model provided by the embodiments of the present application, the model parameter state transition matrix is an unknown quantity.

[0072] Exemplarily, the target state transition matrix can be directly calculated by the state enhancement model based on the state observation sequence, in combination with the initial state transition matrix corresponding to the current moment and the emission matrix; or the target state transition matrix can be obtained by the state enhancement model based on the state observation sequence, in combination with the initial state transition matrix corresponding to the current moment and the emission matrix, by iteratively updating the initial state transition matrix, and the embodiments of the present application do not limit this, which can be determined according to actual application requirements. It can be understood that the DMS outputs the state observation result of the driver at each time point at a specific frequency, and the state observation result output by the DMS can be inaccurate, but different outputs can be generated with a certain probability. In the embodiments of the present application, the emission matrix is constructed to describe the probability distribution of the state observation result output by the DMS under different hidden states (the real state of the driver) through these certain probabilities.

[0073] Exemplarily, the emission matrix can be represented by the following formula:

[0074]

[0075] Where B represents the emission matrix, This represents an element in emission matrix B. This indicates that the state observation result at time n is "awake". This indicates that the state observation result at time n is fatigue. This represents the probability that the observed state at time n is conscious, and the hidden state is conscious. Let represent the probability that the observed state at time n is fatigue, and the hidden state is wakefulness. Let represent the probability that the observed state at time n is conscious and the hidden state is fatigued. This represents the probability that the observed state at time n is fatigue, and the hidden state is fatigue.

[0076] For example, the emission matrix can be a The matrix.

[0077] For example, the transmission matrix provided in this application embodiment can be obtained through statistical analysis based on vehicle driving data, combined with the driver state observation results output by DMS and the corresponding driver real state, that is, statistically analyzing the probability distribution of each state observation result output by DMS under different real driver states.

[0078] It is understood that in the state enhancement model provided in this application embodiment, the model parameter emission matrix is ​​a known quantity.

[0079] In the state enhancement model provided in this application embodiment, a state transition matrix is ​​constructed based on the dynamic evolution characteristics of the driver's true state, and an emission matrix is ​​constructed based on historical state observation results. This enables the model parameters of the state enhancement model to quantify the correlation between state observation results and driver state changes, providing a mathematical basis for the subsequent derivation of hidden states. This achieves an accurate description of the dynamic changes in the driver's state and improves the accuracy and robustness of driver state estimation.

[0080] It is understood that the driver state estimation method provided in this application uses a state enhancement model that includes a state transition matrix and an emission matrix to make the temporal evolution of the driver state follow a hidden Markov process.

[0081] Figure 5 A schematic diagram of a hidden Markov process provided for an embodiment of this application. For example... Figure 5 As shown, This indicates the driver's hidden state at time 0. This indicates the driver's hidden state at moment 1, and so on. This represents the driver's hidden state at time N. This represents the state observation result at time 0. denotes the state observation result at the first time, and the like, denotes the state observation result at the Nth time.

[0082] From Figure 5 It can be seen that, in the hidden Markov process provided by the embodiment of the application, the known model parameter, i.e., the emission matrix B, can be constructed through the state observation result output by the DMS, and then the unknown model parameter, i.e., the state transition matrix A, can be solved through the emission matrix, and then the estimation of the driver hidden state, i.e., the real state of the driver, can be realized.

[0083] For example, the target hidden state probability distribution can be the hidden state probability distribution corresponding to the current time.

[0084] For example, the target hidden state probability distribution can be represented by the following formula:

[0085]

[0086] wherein, denotes the target hidden state probability distribution, N denotes the current time, and i denotes the hidden state of the driver, denotes that the hidden state at the current time N is i, denotes the state observation sequence corresponding to the current time, denotes the target state transition matrix, denotes the target forward probability of the hidden state i at the current time N, denotes the target backward probability of the hidden state i at the current time N, denotes the forward probability of the hidden state being awake at the current time N, denotes the forward probability of the hidden state being fatigue at the current time N.

[0087] For example, the target hidden state probability distribution can be obtained by calculating the product of the target forward probability and the target backward probability corresponding to the current time.

[0088] It can be understood that the target hidden state probability distribution can be the probability distribution of the hidden state being i at the current time N under the condition of the given state observation sequence, the emission matrix and the target state transition matrix, wherein i can be awake or fatigue.

[0089] For example, when i takes the value of 0, i.e. denotes the probability of the hidden state being awake at the current time N, and when i takes the value of 1, i.e. denotes the probability of the hidden state being fatigue at the current time N.

[0090] S403, according to the target hidden state probability distribution, estimating the target driver state corresponding to the current time.

[0091] For example, the target hidden state probability distribution can include a first hidden state probability and a second hidden state probability. The first hidden state probability can be a probability that the hidden state is awake, i.e. The second hidden state probability can be a probability that the hidden state is tired, i.e. .

[0092] In this step, one possible implementation can be: determining whether the first hidden state probability is greater than the second hidden state probability. If the first hidden state probability is greater than the second hidden state probability, it is determined that the hidden state corresponding to the first hidden state probability is the target driver state, such as being awake. If the first hidden state probability is less than the second hidden state probability, it is determined that the hidden state corresponding to the second hidden state probability is the target driver state, such as being tired.

[0093] In the embodiments of the present application, the state observation sequence including at least two state observations corresponding to the current time is obtained, and the state observation sequence is input into the state enhancement model for state enhancement processing. The target hidden state probability distribution of the current time is obtained in combination with the emission matrix and the target state transition matrix. The target driver state corresponding to the current time is estimated according to the target hidden state probability distribution. The target state transition matrix is used to describe the target time sequence evolution rule of the hidden state at the current time. The emission matrix is used to describe the probability distribution of the state observation under different hidden states. In the embodiments of the present application, the state transition matrix used to describe the time sequence evolution rule of the hidden state, i.e., the real state of the driver, is constructed to realize the modeling of the dynamic evolution characteristics of the driver state. The emission matrix used to describe the probability distribution of the state observation under different hidden states is constructed to realize the correlation analysis of the hidden state and the historical state observation. The state enhancement model is further combined with the target state transition matrix and the emission matrix to perform state enhancement processing on the state observation sequence corresponding to the current time, to realize the correction of the state observation of the current time, and to improve the accuracy of the driver state estimation.

[0094] The following will be described in detail with reference to the specific implementation of the step S402 of inputting the state observation sequence into the state enhancement model for state enhancement processing, in combination with the emission matrix and the target state transition matrix to obtain the target hidden state probability distribution corresponding to the current time. Figure 6

[0095] Figure 6 The flowchart of the driver state estimation method provided by the embodiments of the present application is shown in Figure 2 . As shown in Figure 6 ​As shown, in the driver state estimation method, the state observation sequence is input into the state augmentation model for state augmentation processing, and a specific implementation manner of obtaining the target hidden state probability distribution corresponding to the current time in combination with the emission matrix and the target state transition matrix can include the following steps:

[0096] In S601, the state observation sequence is input into the state augmentation model, and the target forward probability and the target backward probability corresponding to the current time are calculated in combination with the emission matrix, the target state transition matrix and the preset hidden state set.

[0097] It can be understood that the state transition matrix in the state augmentation model provided by the embodiment of the present application is an unknown quantity, and therefore the state transition matrix corresponding to each time is calculated or iteratively optimized according to the state observation sequence corresponding to the current time.

[0098] Exemplarily, the preset hidden state set can be a set containing the hidden state as awake or fatigue. For example, when the hidden state i takes the value 0, it indicates that the hidden state is awake, and when the hidden state i takes the value 1, it indicates that the hidden state is fatigue.

[0099] Exemplarily, the target forward probability can be represented by the following formula:

[0100]

[0101] wherein, denotes the target forward probability corresponding to the current time N.

[0102] It can be understood that the target forward probability can be the joint probability of observing the first N state observation results and the hidden state being i at the current time N, under the condition that the target state transition matrix and the emission matrix are known.

[0103] Exemplarily, the target backward probability can be represented by the following formula:

[0104]

[0105] wherein, denotes the target backward probability corresponding to the current time N.

[0106] It can be understood that the target backward probability can be the joint probability of the hidden state being i at the current time N, under the condition that the target state transition matrix, the emission matrix and the hidden state i corresponding to the current time N are known.

[0107] It can be understood that in the driver state estimation method provided by the embodiment of the present application, the target backward probability corresponding to the current time can be 1.

[0108] Optionally, in this step, one possible implementation manner can be: inputting the state observation sequence into the state enhancement model, adopting the forward algorithm, combining the emission matrix, the target state transition matrix and the preset hidden state set to calculate the target forward probability; adopting the backward algorithm, combining the emission matrix, the target state transition matrix and the preset hidden state set to calculate the target backward probability.

[0109] The specific calculation manner of the target forward probability and the target backward probability is similar to the above, which will not be repeated here.

[0110] It should be noted that the calculation manner of the target forward probability and the target backward probability described above is only an example, and there can be multiple calculation manners of calculating the target forward probability by adopting the forward algorithm and calculating the target backward probability by adopting the backward algorithm, and the specific calculation manner of calculating the target forward probability by adopting the forward algorithm and calculating the target backward probability by adopting the backward algorithm is not limited by the embodiments of the present application, and can be determined according to actual application requirements.

[0111] S602, according to the target forward probability and the target backward probability, calculating the target hidden state probability distribution.

[0112] The specific calculation manner of the target hidden state probability distribution is similar to the above, which will not be repeated here.

[0113] According to the embodiments of the present application, by inputting the state observation sequence into the state enhancement model, combining the emission matrix, the target state transition matrix and the preset hidden state set, the target forward probability and the target backward probability corresponding to the current time are calculated, and the target hidden state probability distribution is calculated according to the target forward probability and the target backward probability, the capturing ability of the progressive change of the fatigue state is improved, the joint modeling of the state transition characteristics of the historical state observation results is realized, the accuracy of the driver state estimation is improved, and the robustness of the driver state derivation can be optimized.

[0114] Figure 7 Flowchart of the driver state estimation method provided by the embodiments of the present application Figure 3 As shown in Figure 7 , a specific implementation manner of the driver state estimation method can include the following steps:

[0115] S701, obtaining a state observation sequence corresponding to the current time, the state observation sequence including at least two state observation results.

[0116] The specific implementation manner is similar to the above, which will not be repeated here.

[0117] S702, determining whether the state probability representation coefficient corresponding to the current time converges.

[0118] For example, the state probability representation coefficient can be determined by normalizing or weightedly fusing the forward probability corresponding to the hidden state being fatigue and the forward probability corresponding to the hidden state being sober.

[0119] For example, the state probability representation coefficient corresponding to the current moment can be the state probability representation coefficient corresponding to the last moment adjacent to the current moment which has converged.

[0120] In this step, a possible implementation manner can be as follows: it is determined whether the state probability representation coefficient corresponding to the current moment converges, when the absolute difference between the state probability representation coefficient corresponding to the current moment and the state probability representation coefficient corresponding to the last moment adjacent to the current moment corresponding to the state observation sequence is less than or equal to a preset threshold value, it is determined that the state probability representation coefficient corresponding to the current moment converges, and when the difference between the state probability representation coefficient corresponding to the current moment and the state probability representation coefficient corresponding to the adjacent moment is greater than the preset threshold value, it is determined that the state probability representation coefficient corresponding to the current moment does not converge. The size of the preset threshold value is not limited in the embodiment of the application, and can be determined according to actual application requirements.

[0121] S703, when the state probability representation coefficient does not converge, the state enhancement model performs state enhancement processing based on the state observation sequence, and combines the emission matrix and the initial state transition matrix of the current moment to calculate the hidden state probability distribution and the state transition probability distribution of any moment n, where n is the time index of the state observation sequence.

[0122] It can be understood that when the current moment is the Nth moment, the state observation sequence corresponding to the current moment is composed of the state observation results of the 0th moment, the 1st moment, …, and the N-1th moment, that is, the state observation results of N moments, where any moment n satisfies 0≤n≤N-1.

[0123] Optionally, in this step, a possible implementation manner can include the following steps.

[0124] S7031, the forward probability and the first backward probability of any moment n are calculated according to the state observation sequence, the initial state transition matrix, the emission matrix and the preset hidden state set.

[0125] For example, when the current moment is the moment when the driver state estimation is performed for the first time, the initial state transition matrix can be determined by using an empirical value, for example, the initial state transition matrix can be represented as:

[0126]

[0127] For example, when the current time is not the time when the driver's state is first estimated, the initial state transition matrix can be the target state transition matrix corresponding to the previous time adjacent to the current time. The representation of the target state transition matrix corresponding to the previous time is similar to the representation of the state transition matrix mentioned above, and will not be repeated here.

[0128] For example, the forward probability at any time n can be expressed by the following formula:

[0129]

[0130] in, This represents the forward probability at any time n. This represents the state transition matrix at time n-1.

[0131] For example, when n is 0, which corresponds to the initial time of the state observation sequence, Let n be the initial state transition matrix corresponding to the current time step. When n is 1, Let be the state transition matrix corresponding to time 0.

[0132] It is understandable that the forward probability at any time n can be the result of observing the first n states, given that the initial state transition matrix and emission matrix are known. The joint probability when the hidden state of n is i at any time.

[0133] For example, the first backward probability at any time n can be expressed by the following formula:

[0134]

[0135] in, Let n represent the first backward probability at any time n.

[0136] It is understandable that the backward probability at any time n can be the result of observing the (n+1)th to (N-1)th states, given that the initial state transition matrix, emission matrix, and the hidden state i at any time n are known. The joint probability.

[0137] For example, n can take values ​​from 0 to N-2.

[0138] S7032, calculate the hidden state probability distribution at any time n based on the forward probability and the first backward probability.

[0139] For example, the probability distribution of the hidden state at any time n can be represented by the following formula:

[0140]

[0141] wherein, represents the hidden state probability distribution at any time n.

[0142] For example, the hidden state probability distribution at any time n can be obtained by calculating the product of the forward probability corresponding to any time n and the first backward probability corresponding to any time n.

[0143] It can be understood that the hidden state probability distribution at any time n can be the probability distribution of the corresponding hidden state i at any time n under the condition of a given state observation sequence, emission matrix and initial state transition matrix, wherein i can be awake or fatigue, such as i taking the value of 0 indicating awake and i taking the value of 1 indicating fatigue.

[0144] S7033, according to the forward probability, the initial state transition matrix, the emission matrix, the second backward probability at time n+1 and the state probability representation coefficient, the state transition probability distribution at any time n is calculated.

[0145] For example, the state transition probability distribution at any time n can be represented by the following formula:

[0146]

[0147] wherein, represents the state transition probability distribution at any time n, i and j represent the hidden state of the driver, represents the element in the state transition matrix , represents the probability of transition from the hidden state i to the hidden state j, represents the element in the emission matrix B, represents the probability of the hidden state being j and the state observation result being represents the second backward probability at time n+1 when the hidden state is j, q represents the state probability representation coefficient.

[0148] For example, the state probability representation coefficient can be represented by the following formula:

[0149]

[0150] wherein, represents the forward probability at any time n when the hidden state is awake, represents the forward probability at any time n when the hidden state is fatigue.

[0151] ​It can be understood that the state transition probability distribution at any time n can be the state transition probability distribution when the hidden state at the n+1 time is j and the hidden state at the n time is i, under the condition that the state observation sequence, the emission matrix and the initial state transition matrix are known.

[0152] For example, when the value of j is 0, the corresponding hidden state can be awake, and when the value of j is 1, the corresponding hidden state can be fatigue.

[0153] In S704, the initial state transition matrix is calculated and updated according to the hidden state probability distribution at any time n and the state transition probability distribution, until the state probability representation coefficient converges, to obtain the target state transition matrix.

[0154] For example, the update of the initial state transition matrix can be represented by the following formula:

[0155]

[0156] For example, when the change of the state probability representation coefficient is less than a preset threshold value in the adjacent two update processes, it indicates that the state probability representation coefficient converges. The preset threshold value is not limited in the embodiments of the present application, and can be determined according to actual application requirements.

[0157] It can be understood that the target state transition matrix can be the state transition matrix obtained by iterative optimization when the state probability representation coefficient converges.

[0158] Optionally, when the state probability representation coefficient does not converge, the steps S702 to S704 are repeatedly executed, and the state transition matrix obtained by this iteration optimization is further iteratively optimized until the state probability representation coefficient converges.

[0159] In S705, the state observation sequence is input into the state enhancement model for state enhancement processing, and the target hidden state probability distribution corresponding to the current time is obtained in combination with the emission matrix and the target state transition matrix.

[0160] The specific implementation is similar to the above, and will not be repeated here.

[0161] In S706, the target driver state corresponding to the current time is estimated according to the target hidden state probability distribution.

[0162] The specific implementation is similar to the above, and will not be repeated here.

[0163] The embodiment of the present application determines whether the state probability representation coefficient corresponding to the current moment converges, and when the state probability representation coefficient does not converge, the state enhancement model performs state enhancement processing based on the state observation sequence, combines the emission matrix and the initial state transition matrix of the current moment, calculates the hidden state probability distribution and the state transition probability distribution of any moment n, and further calculates and updates the initial state transition matrix according to the hidden state probability distribution and the state transition probability distribution of any moment n until the state probability representation coefficient converges, so as to obtain the target state transition matrix, improve the adaptability of the state transition matrix to the time sequence evolution rule of the real state of the driver, enhance the dynamic correlation of the driver state estimation, and further improve the estimation accuracy of the driver state.

[0164] Optionally, one possible implementation manner of acquiring the state observation sequence of the current moment in step S401 can be: acquiring the driver face image corresponding to each moment in a target period, the target period being a period corresponding to the start moment and the current moment; estimating the state observation result corresponding to each moment according to the driver face image corresponding to each moment; and arranging all the state observation results corresponding to the target period in time sequence to obtain the state observation sequence.

[0165] Illustratively, the start moment can be the time point corresponding to the first state observation result output by the DMS.

[0166] In one possible implementation manner, the driver face image corresponding to each moment in the target period is collected by a camera installed in the cockpit, the DMS acquires the driver face image corresponding to each moment, extracts the face key features such as the eye aspect ratio, mouth opening degree and head posture of the driver from the driver face image corresponding to each moment based on an image processing algorithm, then estimates the state observation result corresponding to each moment based on the face key features corresponding to each moment, and outputs the state observation result corresponding to each moment, arranges all the state observation results corresponding to the target period in time sequence to obtain the state observation sequence corresponding to the current moment.

[0167] In summary, Figure 8 A comparison diagram of the driver states obtained by different estimation manners provided by the embodiment of the present application is shown. In this example, it is assumed that the false alarm and the missed alarm of the DMS are both set to 5%. As shown in FIG. 8, Figure 8 The horizontal coordinate represents time, which can be in min, and the vertical coordinate represents the driver state. The rectangular box 81 represents the real state of the driver, the rectangular box 82 represents the driver state output by the DMS, and the rectangular box 83 represents the driver state output by the driver state estimation system provided by the embodiment of the present application. From Figure 8As can be seen from the above, compared with the real state of the driver, the driver state output by the DMS has a large number of false positives, and the driver state output by the driving state estimation system provided in the embodiment of the application has higher accuracy.

[0168] From Figure 8 As can be seen from the above, the driver state estimation method provided in the embodiment of the application can realize correction on the inaccurate state observation results output by the DMS, and further estimate the driver state that is closer to the real state of the driver and has higher accuracy.

[0169] The following is an apparatus embodiment of the application, which can be used to execute the method embodiments of the application. For details not disclosed in the apparatus embodiments of the application, refer to the method embodiments of the application.

[0170] Figure 9 The structure diagram of the driver state estimation apparatus provided in the embodiment of the application is shown in FIG. 9. As shown in FIG. 9, the driver state estimation apparatus 90 includes an acquisition module 910, an enhancement processing module 920, and an estimation module 930. Figure 9

[0171] The acquisition module 910 is configured to acquire a state observation sequence corresponding to a current time, the state observation sequence including at least two state observation results.

[0172] The enhancement processing module 920 is configured to input the state observation sequence into a state enhancement model for state enhancement processing, and obtain a target hidden state probability distribution corresponding to the current time in combination with an emission matrix and a target state transition matrix, the target state transition matrix being used to describe a target time sequence evolution rule of the hidden state at the current time, and the emission matrix being used to describe a probability distribution of the state observation result under different hidden states.

[0173] The estimation module 930 is configured to estimate a target driver state corresponding to the current time according to the target hidden state probability distribution.

[0174] In a possible implementation, the enhancement processing module 920 is specifically configured to input the state observation sequence into the state enhancement model, calculate the target forward probability and the target backward probability in combination with the emission matrix, the target state transition matrix, and a preset hidden state set corresponding to the current time, and calculate the target hidden state probability distribution according to the target forward probability and the target backward probability.

[0175] In a possible implementation, the enhancement processing module 920 is further configured to input the state observation sequence into the state enhancement model, calculate the target forward probability by using a forward algorithm in combination with the emission matrix, the target state transition matrix, and the preset hidden state set, and calculate the target backward probability by using a backward algorithm in combination with the emission matrix, the target state transition matrix, and the preset hidden state set. ​

[0176] In a possible implementation, before the target hidden state probability distribution corresponding to the current time is obtained by combining the emission matrix and the target state transition matrix, the driver state estimation apparatus further includes an updating module (not shown in the figure), which is specifically configured to: determine whether the state probability representation coefficient corresponding to the current time converges; when the state probability representation coefficient does not converge, the state enhancement model performs state enhancement processing based on the state observation sequence, combines the emission matrix and the initial state transition matrix of the current time, and calculates the hidden state probability distribution and the state transition probability distribution at any time n, where n is the time index of the state observation sequence; and according to the hidden state probability distribution and the state transition probability distribution at any time n, the initial state transition matrix is calculated and updated until the state probability representation coefficient converges, and the target state transition matrix is obtained.

[0177] In a possible implementation, the updating module is further configured to: calculate the forward probability and the first backward probability at any time n according to the state observation sequence, the initial state transition matrix corresponding to the current time, the emission matrix, and the preset hidden state set; calculate the hidden state probability distribution at any time n according to the forward probability and the first backward probability; and calculate the state transition probability distribution at any time n according to the forward probability, the initial state transition matrix, the emission matrix, the second backward probability at time n+1, and the state probability representation coefficient.

[0178] In a possible implementation, the obtaining module 910 is specifically configured to: obtain the driver face image corresponding to each time in a target time period, where the target time period is a time period corresponding to the starting time and the current time; estimate the state observation result corresponding to each time according to the driver face image corresponding to each time; and arrange all the state observation results corresponding to the target time period in time sequence to obtain the state observation sequence.

[0179] The driver state estimation apparatus provided in this embodiment can execute the method provided in the method embodiment, and has similar implementation principles and technical effects. Details are not described herein again.

[0180] The embodiment of the present application further provides a vehicle including a vehicle body and a driver state estimation system configured to implement the method provided in the method embodiment. The driver state estimation system is shown in the above Figure 3 embodiment, and details are not described herein again.

[0181] Figure 10 The structural schematic diagram of an electronic device provided in an embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the electronic device includes a processor 100, a memory 200, a communication interface 300, and a power supply 400. Figure 10As shown, the electronic device 10 provided by the embodiment includes at least one processor 101 and a memory 102. Optionally, the device 10 further includes a communication component 103. The processor 101, the memory 102 and the communication component 103 are connected through a bus 104.

[0182] In the implementation process, the at least one processor 101 executes the computer execution instructions stored in the memory 102, so that the at least one processor 101 executes the above-mentioned method.

[0183] The specific implementation process of the processor 101 can refer to the above-mentioned method embodiment, which has similar implementation principles and technical effects, and will not be described here in detail.

[0184] In the above-mentioned embodiments, it should be understood that the processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC) and the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor and the like. The steps of the method disclosed in the application can be directly embodied as execution completed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0185] The memory can include a random access memory (RAM), and can also include a non-volatile memory (NVM), for example, at least one disk memory.

[0186] The bus can be an industry standard architecture (ISA) bus, a peripheral component (PCI) bus or an extended industry standard architecture (EISA) bus and the like. The bus can be divided into an address bus, a data bus, a control bus and the like. For the convenience of representation, the bus in the drawings of the present application does not limit to only one bus or one type of bus.

[0187] The embodiment of the present application further provides a computer program product, including a computer program, which is executed by a processor to realize the above-mentioned method.

[0188] The embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores computer execution instructions.

[0189] The readable storage medium can be realized by any type of volatile or nonvolatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0190] An exemplary readable storage medium is coupled to the processor, so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in the device.

[0191] The division of units is only a logical function division, and in actual implementation, another division mode can be used, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0192] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, can be located in one place, or can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment of the present application.

[0193] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0194] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the method of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0195] It can be understood by those skilled in the art that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The aforementioned program can be stored in a computer readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, and various media that can store program codes.

[0196] Finally, it should be noted that: those skilled in the art will easily think of other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application that follow the general principles of the present application and include common knowledge or conventional technical means in the art that are not disclosed in the present application, and is not limited to the precise structure described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present application is only limited by the appended claims.

Claims

1. A method for estimating driver state, characterized in that, include: Obtain the state observation sequence corresponding to the current time, wherein the state observation sequence includes at least two state observation results; The state observation sequence is input into the state enhancement model for state enhancement processing. By combining the emission matrix and the target state transition matrix, the probability distribution of the target hidden state corresponding to the current time is obtained. The target state transition matrix is ​​used to describe the temporal evolution law of the hidden state of the target at the current time, and the emission matrix is ​​used to describe the probability distribution of the state observation results under different hidden states. Based on the probability distribution of the target's hidden state, the target driver's state at the current moment is estimated.

2. The driver state estimation method according to claim 1, characterized in that, The step of inputting the state observation sequence into the state enhancement model for state enhancement processing, and combining the emission matrix and the target state transition matrix to obtain the target hidden state probability distribution corresponding to the current time moment includes: The state observation sequence is input into the state enhancement model, and the forward probability and backward probability of the target at the current moment are calculated by combining the emission matrix, the target state transition matrix and the preset hidden state set. The target's hidden state probability distribution is calculated based on the target's forward probability and backward probability.

3. The driver state estimation method according to claim 2, characterized in that, The step of inputting the state observation sequence into the state enhancement model, and combining the emission matrix, the target state transition matrix, and the preset hidden state set to calculate the target forward probability and target backward probability at the current moment includes: The state observation sequence is input into the state enhancement model, and the forward algorithm is used to calculate the forward probability of the target by combining the emission matrix, the target state transition matrix and the preset hidden state set. The backward algorithm is used to calculate the backward probability of the target by combining the emission matrix, the target state transition matrix and the preset hidden state set.

4. The driver state estimation method according to any one of claims 1 to 3, characterized in that, Before obtaining the target hidden state probability distribution corresponding to the current moment by combining the emission matrix and the target state transition matrix, the method further includes: Determine whether the state probability representation coefficients corresponding to the current moment have converged; When the state probability representation coefficients do not converge, the state enhancement model performs state enhancement processing based on the state observation sequence, and calculates the hidden state probability distribution and state transition probability distribution at any time n by combining the emission matrix and the initial state transition matrix at the current time, where n is the time index of the state observation sequence. Based on the hidden state probability distribution and state transition probability distribution at any time n, the initial state transition matrix is ​​calculated and updated until the state probability representation coefficients converge, thus obtaining the target state transition matrix.

5. The driver state estimation method according to claim 4, characterized in that, The state enhancement model performs state enhancement processing based on the state observation sequence, combining the emission matrix and the initial state transition matrix at the current time to calculate the hidden state probability distribution and state transition probability distribution at any time n, including: Based on the state observation sequence, the initial state transition matrix, the emission matrix, and the preset hidden state set, calculate the forward probability and the first backward probability at any time n; Based on the forward probability and the first backward probability, calculate the hidden state probability distribution at any time n; The state transition probability distribution at any time n is calculated based on the forward probability, the initial state transition matrix, the emission matrix, the second backward probability at time n+1, and the state probability characterization coefficient.

6. The driver state estimation method according to any one of claims 1 to 3, characterized in that, The process of obtaining the current state observation sequence includes: Acquire the driver's facial image at each moment within a target time period, where the target time period is the time period between the start time and the current time. Based on the driver's facial image at each time step, estimate the state observation result at each time step. Arrange all state observation results corresponding to the target time period in chronological order to obtain the state observation sequence.

7. A driver state estimation device, characterized in that, include: The acquisition module is used to acquire the state observation sequence corresponding to the current time, wherein the state observation sequence includes at least two state observation results; The enhancement processing module is used to input the state observation sequence into the state enhancement model for state enhancement processing, and combine the emission matrix and the target state transition matrix to obtain the probability distribution of the target hidden state at the current time. The target state transition matrix is ​​used to describe the target temporal evolution law of the hidden state at the current time, and the emission matrix is ​​used to describe the probability distribution of the state observation results under different states. The estimation module is used to estimate the target driver state at the current moment based on the target hiding state probability distribution.

8. A vehicle, characterized in that, It includes a vehicle body and a driver state estimation system, wherein the driver state estimation system is used to implement the driver state estimation method as described in any one of claims 1 to 6.

9. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the driver state estimation method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the driver state estimation method as described in any one of claims 1 to 6.

11. A computer program product, characterized in that, include: A computer program, which, when executed by a processor, implements the driver state estimation method as described in any one of claims 1 to 6.

Images