Driver state judgment method and device and driver monitoring system

Abstract

The application provides a driver state judgment method and device and a driver monitoring system. The method obtains a true value and a predicted value of a regression task in a driver monitoring task. Then, a first continuous probability distribution function is determined according to the true value, a second continuous probability distribution function is determined according to the predicted value, a loss function corresponding to the regression task is obtained based on a loss function corresponding to a classification task and the above functions, a trained neural network model is obtained based on the loss function, the neural network model inputs driver data and outputs a predicted value and a confidence of the driver. Thus, the application can determine the prediction result output by the neural network by using the confidence output by the neural network, improve the accuracy of the driver state judgment based on the prediction result output by the neural network, identify the behavior of the driver that is not conducive to the safe driving of the vehicle in time, and greatly reduce the probability of accidents of the vehicle.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of personnel monitoring, and in particular to a driver state judgment method and device and a driver monitoring system. BACKGROUND

[0002] In the driving field, due to certain behaviors of the driver, the safety driving of the vehicle will be affected, therefore, it is necessary to identify the behaviors of the driver, and when the behaviors of the driver are identified as behaviors that are not conducive to the safety driving of the vehicle, timely measures are taken to reduce the influence of the behaviors of the driver on the safety driving of the vehicle as much as possible.

[0003] The existing driver monitoring system (DMS) generally determines the prediction result output by the neural network through the confidence value output by the neural network for the classification task in the driver monitoring task, such as abnormal action recognition (for example, recognizing that the driver smokes or makes a phone call, etc.). For example, when the DMS recognizes that the driver smokes, the confidence value output by the neural network is used to determine whether the prediction result output by the neural network is reliable, for example, the lower the confidence value, the lower the reliability of the prediction result output by the neural network. In this way, in subsequent processing, the prediction result corresponding to the low confidence value will be failed, thereby improving the accuracy of the subsequent judgment of the driver state based on the prediction result output by the neural network.

[0004] However, for the regression task in the driver monitoring task (for example, face key point, gaze, head pose estimation, etc.), the DMS cannot determine whether the prediction result output by the neural network is reliable, that is, the DMS cannot determine the prediction result output by the neural network, thereby resulting in low accuracy of the subsequent judgment of the driver state based on the prediction result output by the neural network, and the driver's behavior that is not conducive to the safety driving of the vehicle cannot be identified in time, greatly increasing the probability of accidents of the vehicle. SUMMARY

[0005] To solve the problems in the prior art, the present application provides a driver state judgment method, device and driver monitoring system.

[0006] In a first aspect, the present application provides a driver state judgment method, which comprises:

[0007] obtaining a true value and a prediction value of a regression task in a driver monitoring task, and determining a first continuous probability distribution function according to the true value and a second continuous probability distribution function according to the prediction value;

[0008] obtain a loss function corresponding to the regression task based on the loss function corresponding to the classification task in the driver monitoring task and the first continuous probability distribution function and the second continuous probability distribution function;

[0009] obtain a trained neural network model based on the loss function corresponding to the regression task, wherein the neural network model inputs first driver data and outputs a predicted value and a confidence parameter of the first driver;

[0010] obtain a predicted value and a confidence parameter of a second driver based on second driver data and the neural network model, determine a confidence degree based on the confidence parameter of the second driver, and judge a state of the second driver according to the confidence degree and the predicted value of the second driver.

[0011] In a possible implementation, the obtaining of the loss function corresponding to the regression task based on the loss function corresponding to the classification task in the driver monitoring task and the first continuous probability distribution function and the second continuous probability distribution function includes:

[0012] obtain a first loss function based on the loss function corresponding to the classification task and the first continuous probability distribution function and the second continuous probability distribution function;

[0013] determine a second loss function according to the first loss function and a true value of the regression task;

[0014] parameterize the second continuous probability distribution function, and obtain the loss function corresponding to the regression task based on the parameterized second continuous probability distribution function and the second loss function.

[0015] In a possible implementation, the parameterization of the second continuous probability distribution function includes:

[0016] parameterize the second continuous probability distribution function based on a Laplace distribution or a Gaussian distribution.

[0017] In a possible implementation, the obtaining of the trained neural network model based on the loss function corresponding to the regression task includes:

[0018] set the loss function corresponding to the regression task as a loss function of a preset initial neural network model;

[0019] input the first driver data into the preset initial neural network model, and calculate a value of the loss function of the preset initial neural network model;

[0020] adjust the preset initial neural network model according to the value of the loss function, so that the value of the loss function of the adjusted preset initial neural network model meets a preset condition;

[0021] obtain the trained neural network model based on the adjusted preset initial neural network model.

[0022] In a possible implementation, before the loss function corresponding to the regression task is set as the loss function of the preset initial neural network model, the method further includes:

[0023] processing the loss function corresponding to the regression task according to a preset neural network model convergence method to obtain a third loss function;

[0024] setting a loss weight for each term in the third loss function respectively to obtain a fourth loss function;

[0025] the loss function corresponding to the regression task is set as the loss function of the preset initial neural network model, including:

[0026] the fourth loss function is set as the loss function of the preset initial neural network model.

[0027] In a possible implementation, the judging the state of the second driver according to the confidence and the predicted value of the second driver includes:

[0028] determining whether the confidence is greater than a preset threshold, the preset threshold being determined according to a minimum value of the confidence corresponding to the predicted value of the driver output by the neural network used for judging the state of the driver;

[0029] if the confidence is greater than the preset threshold, judging the state of the second driver according to the predicted value of the second driver.

[0030] In a possible implementation, the confidence parameter of the second driver is determined according to a standard deviation of the neural network model;

[0031] the confidence is determined based on the confidence parameter of the second driver, including:

[0032] performing nonlinear mapping processing on the confidence parameter of the second driver to obtain the confidence.

[0033] In a possible implementation, the first continuous probability distribution function is a Dirichlet function.

[0034] In a second aspect, an embodiment of the present application provides a driver state judgment device, the device includes:

[0035] a function determining module, configured to obtain a true value and a predicted value of a regression task in the driver monitoring task, determine a first continuous probability distribution function according to the true value, and determine a second continuous probability distribution function according to the predicted value;

[0036] a function obtaining module, configured to obtain a loss function corresponding to the regression task based on a loss function corresponding to a classification task in the driver monitoring task, and the first continuous probability distribution function and the second continuous probability distribution function;

[0037] a model obtaining module, configured to obtain a trained neural network model based on the loss function corresponding to the regression task, wherein the neural network model inputs first driver data and outputs a predicted value and a confidence parameter of the first driver;

[0038] a state determining module, configured to obtain second driver data, obtain a predicted value and a confidence parameter of the second driver based on the second driver data and the neural network model, determine a confidence degree based on the confidence parameter of the second driver, and determine a state of the second driver according to the confidence degree and the predicted value of the second driver.

[0039] In a possible implementation, the function obtaining module is specifically configured to:

[0040] obtain a first loss function based on the loss function corresponding to the classification task, and the first continuous probability distribution function and the second continuous probability distribution function;

[0041] determine a second loss function according to the first loss function and a true value of the regression task;

[0042] parameterize the second continuous probability distribution function, and obtain the loss function corresponding to the regression task based on the parameterized second continuous probability distribution function and the second loss function.

[0043] In a possible implementation, the function obtaining module is specifically configured to:

[0044] parameterize the second continuous probability distribution function based on a Laplace distribution or a Gaussian distribution.

[0045] In a possible implementation, the model obtaining module is specifically configured to:

[0046] set the loss function corresponding to the regression task as a loss function of a preset initial neural network model;

[0047] input the first driver data into the preset initial neural network model, and calculate a value of the loss function of the preset initial neural network model.

[0048] adjust the preset initial neural network model according to the value of the loss function, so that the value of the loss function of the adjusted preset initial neural network model meets a preset condition;

[0049] obtain the trained neural network model based on the adjusted preset initial neural network model,

[0050] In a possible implementation, the model obtaining module is specifically configured to:

[0051] process the loss function corresponding to the regression task according to a preset neural network model convergence method, to obtain a third loss function;

[0052] set a loss weight for each term in the third loss function respectively, to obtain a fourth loss function;

[0053] set the fourth loss function as the loss function of the preset initial neural network model.

[0054] In a possible implementation, the state judging module is specifically configured to:

[0055] judge whether the confidence degree is greater than a preset threshold, the preset threshold being determined according to a minimum value of a confidence degree corresponding to a predicted value of a driver output by a neural network used for judging a state of the driver;

[0056] if the confidence degree is greater than the preset threshold, judge the state of the second driver according to the predicted value of the second driver.

[0057] In a possible implementation, the confidence parameter of the second driver is determined according to a standard deviation of the neural network model.

[0058] The state judging module is specifically configured to:

[0059] perform nonlinear mapping processing on the confidence parameter of the second driver, to obtain the confidence degree.

[0060] In a possible implementation, the first continuous probability distribution function is a Dirichlet function.

[0061] In a third aspect, an embodiment of the present application provides a driver monitoring system, comprising:

[0062] a processor;

[0063] a memory; and

[0064] a computer program;

[0065] The computer program is stored in the memory and is configured to be executed by the processor, and the computer program includes instructions for executing the method according to the first aspect.

[0066] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program causes a server to execute the method according to the first aspect.

[0067] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes computer instructions, and the computer instructions are executed by a processor to execute the method according to the first aspect.

[0068] The driver state judgment method, device and driver monitoring system provided by the embodiments of the present application obtain the true value and the predicted value of the regression task in the driver monitoring task, then determine the first continuous probability distribution function according to the true value, determine the second continuous probability distribution function according to the predicted value, obtain the loss function corresponding to the regression task based on the loss function corresponding to the classification task and the first continuous probability distribution function and the second continuous probability distribution function, and thus obtain the trained neural network model based on the loss function, obtain the predicted value and the confidence of the driver based on the model and the driver data, and judge the state of the driver. The embodiments of the present application solve the problem that the DMS cannot determine the prediction result output by the neural network for the regression task, thereby reducing the accuracy of the prediction result output by the neural network for the subsequent judgment of the state of the driver. Moreover, the embodiments of the present application improve the accuracy of the judgment of the state of the driver, can identify the behavior of the driver that is not conducive to the safe driving of the vehicle in time, and greatly reduce the probability of accidents of the vehicle. BRIEF DESCRIPTION OF DRAWINGS

[0069] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0070] Figure 1 The driver monitoring system architecture schematic diagram provided by the embodiments of the present application is shown in the figure;

[0071] Figure 2 The flowchart of the driver state judgment method provided by the embodiments of the present application is shown in the figure;

[0072] Figure 3 The schematic diagram of training the neural network model provided by the embodiments of the present application is shown in the figure;

[0073] Figure 4 FIG. 1 is a flow diagram of another driver state determination method according to an embodiment of the present application;

[0074] Figure 5 FIG. 2 is a structural diagram of a driver state determination device according to an embodiment of the present application;

[0075] Figure 6 FIG. 3 is a basic hardware architecture diagram of a driver monitoring system according to an embodiment of the present application. DETAILED DESCRIPTION

[0076] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0077] The terms "first", "second", "third", and "fourth" and the like in the description, claims, and drawings of the present application (if any) are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in other than the order illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a list of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to such processes, methods, products, or devices.

[0078] The existing DMS determines the prediction result output by the neural network through the confidence of the neural network output for the classification task in the driver monitoring task, such as abnormal action recognition (usually defined as a multi-classification task, including smoking, making a phone call, etc.). For example, the loss function of abnormal action recognition is a cross-entropy loss function, the true value is a one-hot discrete probability distribution p, and the prediction is a discrete probability distribution q, and the loss function can be expressed as:

[0079]

[0080] The neural network based on the loss function outputs a prediction result and a confidence value. The lower the confidence value, the lower the reliability of the prediction result output by the neural network. Thus, in subsequent processing, the prediction result corresponding to the low confidence value is failed, thereby improving the accuracy of subsequent judgment of the driver state based on the prediction result output by the neural network

[0081] However, for the regression task in the driver monitoring task (for example, face key point, gaze, head pose estimation, etc.), the loss function is generally L1 / L2, norm or smoothL1 function, as follows:

[0082]

[0083] Wherein, the true value p is a specific numerical value (such as face corner coordinate, head pose Euler angle, etc.), q is the predicted value, and β is a known parameter which can be pre-set by relevant personnel according to experience. The neural network based on the loss function outputs a prediction result. The DMS cannot determine whether the prediction result output by the neural network is reliable, that is, cannot determine the prediction result output by the neural network, thereby resulting in low accuracy of subsequent judgment of the driver state based on the prediction result output by the neural network, and failing to timely identify the behavior of the driver that is not conducive to the safe driving of the vehicle, greatly increasing the probability of vehicle accidents.

[0084] To solve the above problems, the embodiment of the present application proposes a driver state judgment method, based on the loss function corresponding to the classification task in the driver monitoring task, a loss function corresponding to the regression task is constructed, thereby obtaining a trained neural network model based on the loss function corresponding to the regression task, wherein the neural network model inputs driver data and outputs the predicted value and confidence of the driver, thereby the embodiment of the present application can determine the prediction result output by the above neural network through the confidence output by the above neural network, improve the accuracy of subsequent judgment of the driver state based on the prediction result output by the above neural network, timely identify the behavior of the driver that is not conducive to the safe driving of the vehicle, and greatly reduce the probability of vehicle accidents.

[0085] Optionally, the driver state judgment method provided by the embodiment of the present application can be applied to the driver monitoring system as shown in Figure 1 In Figure 1For example, in order to detect whether the driver's line of sight is normal, the driver monitoring system architecture can include a processing device 11 and a plurality of acquisition units. Here, the plurality of acquisition units are exemplified by a first acquisition unit 12 and a second acquisition unit 13. The first acquisition unit 12 can be arranged in a first vehicle, and the second acquisition unit 13 can be arranged in a second vehicle. The first acquisition unit 12 acquires data of a driver in the first vehicle, such as a face image of the driver. Similarly, the second acquisition unit 13 acquires data of a driver in the second vehicle. Exemplarily, the first acquisition unit 12 and the second acquisition unit 13 can be cameras.

[0086] It can be understood that the structure exemplified by the embodiments of the present application does not constitute a specific limitation on the driver state judgment architecture. In other possible embodiments of the present application, the above-mentioned architecture can include more or fewer components than the illustration, or combine certain components, or split certain components, or different component arrangement, which can be determined according to actual application scenarios, and is not limited herein. Figure 1 The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0087] In the specific implementation process, the first acquisition unit 12 can acquire a face image of a driver in the first vehicle after the driver enters the vehicle and starts driving, and send the acquired image to the processing device 11. Similarly, the second acquisition unit 13 can acquire a face image of a driver in the second vehicle after the driver enters the vehicle and starts driving, and send the acquired image to the processing device 11.

[0088] The processing device 11 can construct a loss function corresponding to the task of detecting whether the driver's line of sight is normal based on a loss function corresponding to a classification task in the driver monitoring task, for example, a loss function corresponding to the task of identifying the driver smoking. Thus, based on the loss function corresponding to the regression task, a trained neural network model is obtained, wherein the neural network model inputs driver data, such as the face image of the driver sent by the first acquisition unit 12 and the second acquisition unit 13, and outputs the predicted value and the confidence of the driver, such as whether the driver's line of sight is normal and the confidence, so that the processing device 11 can determine the prediction result (e.g., whether the driver's line of sight is normal) output by the neural network through the confidence output by the neural network. As mentioned above, the lower the confidence value, the lower the credibility of the prediction result output by the neural network. In this way, in subsequent processing, the prediction result corresponding to the low confidence will be failed, thereby improving the accuracy of the subsequent judgment of the driver state based on the prediction result output by the neural network.

[0089] Furthermore, the aforementioned architecture may also include a reminder unit, which can remind the driver when it is determined that the driver's condition is detrimental to the safe operation of the vehicle. The reminder unit can be installed in the vehicle; for example, taking the aforementioned architecture as an example with two reminder units, one reminder unit can be installed in each of the first and second vehicles. The processing device 11 can determine the driver's condition in the first and second vehicles based on the prediction results output by the aforementioned neural network. If it is determined that the driver's condition in the first vehicle is detrimental to the safe operation of the vehicle, it can send a reminder message to the reminder unit in the first vehicle. The reminder unit in the first vehicle then reminds the driver based on the reminder message, for example, by playing the reminder message aloud.

[0090] The architecture described above may also include a display unit, which can be used to display the driver's predicted values ​​and confidence levels.

[0091] The display unit can also be a touch screen, used to receive user commands while displaying the above content, so as to realize interaction with the user.

[0092] It should be understood that the above-mentioned processing device can be implemented by a processor reading instructions from memory and executing those instructions, or it can be implemented by a chip circuit.

[0093] The above system is only an example system. In specific implementation, it can be set up according to application requirements.

[0094] It is understood that the system architecture described in the embodiments of this application is for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and does not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0095] The technical solutions of this application are described below using several embodiments as examples. The same or similar concepts or processes may not be repeated in some embodiments.

[0096] Figure 2 This is a flowchart illustrating a driver status determination method provided in an embodiment of this application. The execution entity in this embodiment can be... Figure 1 The processing device in the process can be specifically executed based on the actual application scenario, and this application embodiment does not impose any particular restrictions on this. For example Figure 2 As shown, the driver status determination method provided in this application embodiment may include the following steps:

[0097] S201: Obtain the true value and predicted value of the regression task in the driver monitoring task, and determine the first continuous probability distribution function based on the true value and the second continuous probability distribution function based on the predicted value.

[0098] The first continuous probability distribution function can be a Dirac function.

[0099] As known from the above, for a regression task in a driver monitoring task (such as face key point, gaze, head pose estimation, etc.), the loss function is generally L1 / L2, norm or smoothL1 function, such as:

[0100] a(p, q) = SmoothL1(p, q)

[0101] The processing device can determine the first continuous probability distribution function p = S(a) according to the true value p, and determine the second continuous probability distribution function according to the predicted value q. Wherein, a is the true value.

[0102] S202: Based on the loss function corresponding to the classification task in the driver monitoring task, and the first continuous probability distribution function and the second continuous probability distribution function, the loss function corresponding to the regression task is obtained.

[0103] Here, the processing device can substitute the first continuous probability distribution function and the second continuous probability distribution function into the loss function corresponding to the classification task, such as the above Thus, the first loss function is obtained:

[0104] a(p, q) = -∫p(x)logq(x)

[0105] Then, the processing device can determine the second loss function according to the first loss function and the true value of the regression task:

[0106] a(p, q) = -logq(a)

[0107] Further, the processing device parameterizes the second continuous probability distribution function, for example, parameterizes the second continuous probability distribution function based on Laplace distribution or Gaussian distribution.

[0108] Gaussian distribution:

[0109] Wherein, m is the mean, and σ is the standard deviation

[0110] Gaussian distribution:

[0111] Here, taking the Gaussian distribution as an example, the processing device obtains the loss function corresponding to the regression task based on the parameterized second continuous probability distribution function and the second loss function, for example, substitutes the parameterized second continuous probability distribution function into the second loss function to obtain:

[0112]

[0113] S203: Based on the loss function corresponding to the above regression task, obtain a trained neural network model, wherein the above neural network model takes the first driver's data as input and outputs the first driver's predicted value and confidence parameter.

[0114] For example, the processing device can set the loss function corresponding to the regression task as the loss function of the preset initial neural network model, thereby inputting the first driver data into the preset initial neural network model, calculating the value of the loss function of the preset initial neural network model, and adjusting the preset initial neural network model according to the value of the loss function so that the value of the loss function of the adjusted preset initial neural network model meets the preset conditions. Finally, based on the adjusted preset initial neural network model, the trained neural network model is obtained.

[0115] For example, such as Figure 3 As shown, the processing device can generate a training sample set based on the first driver data, and then use the training sample set to train the preset initial neural network model in batches, calculating the loss function value of the preset initial neural network model. When the loss function value meets a preset condition, the processing device can calculate the gradient corresponding to the initial parameters of the preset initial neural network model based on the loss function value, and then update the initial parameters of the preset initial neural network model based on the gradient to obtain the trained neural network model. The preset initial neural network model can be set according to actual conditions; for example, the processing device can set the neural network model corresponding to the classification task as the preset initial neural network model. The preset conditions can also be determined according to actual conditions, for example, based on the loss function value when the model loss is small and the model is stable.

[0116] Furthermore, before setting the loss function corresponding to the regression task as the loss function of the preset initial neural network model, the processing device can also process the loss function corresponding to the regression task according to the preset neural network model convergence method to obtain a third loss function. For example,

[0117] Let α = log(σ) 2 )and thus obtain Substituting this into the second loss function above, we obtain the third loss function:

[0118]

[0119] The third loss function is the sum of two equations. Primarily used to learn the mean / harmonized standard deviation. Mainly used to learn the standard deviation.

[0120] Then, the processing device can set a loss weight for each term in the third loss function respectively, and obtain a fourth loss function:

[0121]

[0122] Wherein, w1 and w2 represent the loss weight, which can be obtained according to actual conditions, for example, the loss weight adjusted and set by relevant technical personnel according to the output of the preset initial neural network model, or the gradient of the preset initial neural network model.

[0123] Correspondingly, the processing device can set the fourth loss function as the loss function of the preset initial neural network model, and re-execute the step of inputting the first driver data into the preset initial neural network model, calculating the value of the loss function of the preset initial neural network model, adjusting the preset initial neural network model according to the value of the loss function, and obtaining the trained neural network model based on the adjusted preset initial neural network model. In the neural network structure design, the parameters m and a of the fourth loss function after parameterization can be realized by full connection.

[0124] Here, the processing device considers the differences between each term in the third loss function, respectively applies a loss weight to each term of the loss function for the sample, so that the loss weight of each term of the loss function is different, and the loss function is better adjusted subsequently to obtain a trained neural network model.

[0125] S204: Obtain second driver data, obtain the predicted value and confidence parameter of the second driver based on the second driver data and the neural network model, determine the confidence degree based on the confidence parameter of the second driver, and judge the state of the second driver according to the confidence degree and the predicted value of the second driver.

[0126] Here, the confidence parameter is determined according to the standard deviation of the neural network model. When predicting the driver's memory based on the neural network, the processing device can directly take the values of m and a in the fourth loss function from the full connection neurons, m is the numerical result, a is converted into σ to obtain the standard deviation, and then the confidence degree can be obtained by some nonlinear mapping in the interval [0, 1]. Further, the processing device can judge the state of the second driver according to the confidence degree and the predicted value of the second driver.

[0127] The embodiment of the application obtains the true value and the predicted value of the regression task in the driver monitoring task, and then determines the first continuous probability distribution function according to the true value and determines the second continuous probability distribution function according to the predicted value. The loss function corresponding to the regression task is obtained based on the loss function corresponding to the classification task and the first continuous probability distribution function and the second continuous probability distribution function. Thus, the trained neural network model is obtained based on the loss function, and the predicted value and the confidence of the driver are obtained based on the model and the driver data, so that the state of the driver is judged, and the problem that the DMS cannot judge the prediction result output by the neural network for the regression task is solved, thereby greatly reducing the probability of accidents of the vehicle.

[0128] In addition, when judging the state of the second driver according to the confidence and the predicted value of the second driver, the processing device also considers whether the confidence is greater than a preset threshold. If the confidence is greater than the preset threshold, the processing device can judge the state of the second driver according to the predicted value of the second driver. The preset threshold is determined according to the minimum value of the confidence corresponding to the predicted value of the driver output by the neural network used for judging the state of the driver. When the confidence is greater than the preset threshold, it means that the predicted value of the second driver output by the neural network has high credibility, that is, high accuracy, and the processing device judges the state of the second driver according to the predicted value of the second driver, so that the accuracy of the obtained judgment result is high and meets the needs of practical applications. Figure 4 Another flowchart of a method for judging the state of a driver is provided for the embodiment of the application. As shown in Figure 4 The method comprises the following steps.

[0129] S401: Obtain the true value and the predicted value of the regression task in the driver monitoring task, and determine the first continuous probability distribution function according to the true value and determine the second continuous probability distribution function according to the predicted value.

[0130] S402: Obtain the loss function corresponding to the regression task based on the loss function corresponding to the classification task in the driver monitoring task and the first continuous probability distribution function and the second continuous probability distribution function.

[0131] S403: Obtain the trained neural network model based on the loss function corresponding to the regression task, wherein the neural network model inputs the first driver data and outputs the predicted value and the confidence parameter of the first driver.

[0132] The steps S401-S403 have the same implementation manner as the steps S201-S203, and details are not described herein.

[0133] S404: Obtain second driver data, obtain a predicted value and a confidence parameter of the second driver based on the second driver data and the neural network model, and determine a confidence degree based on the confidence parameter of the second driver, and determine whether the confidence degree is greater than a preset threshold.

[0134] The preset threshold is determined according to a minimum value of the confidence degree corresponding to the predicted value of the driver output by the neural network for judging the state of the driver.

[0135] The step S404 has the same content as the step S204, and details are not described herein.

[0136] S405: If the confidence degree is greater than the preset threshold, the state of the second driver is determined according to the predicted value of the second driver.

[0137] Here, when the confidence degree is greater than the preset threshold, it means that the predicted value of the second driver output by the neural network has a high credibility, that is, a high accuracy, otherwise, it means that the predicted value of the second driver output by the neural network has a low credibility, that is, a low accuracy.

[0138] In order to improve the accuracy of the subsequent driver state judgment, the processing device can determine the state of the second driver according to the predicted value of the second driver when the confidence degree is greater than the preset threshold, and discard the predicted value of the second driver and not determine the state of the second driver when the confidence degree is less than or equal to the preset threshold.

[0139] In the embodiment of the present application, when judging the state of the second driver according to the confidence and the predicted value of the second driver, the processing device also considers whether the confidence is greater than a preset threshold. When the confidence is greater than the preset threshold, it means that the predicted value of the second driver output by the neural network has a high credibility, i.e., high accuracy, and the processing device can judge the state of the second driver according to the predicted value of the second driver, thereby improving the accuracy of the judgment result. The processing device can also construct a loss function corresponding to a regression task based on a loss function corresponding to a classification task in the driver monitoring task, thereby obtaining a trained neural network model based on the loss function corresponding to the regression task, wherein the neural network model inputs driver data and outputs a predicted value and a confidence of the driver, so that the embodiment of the present application can determine the prediction result output by the neural network through the confidence output by the neural network, improve the accuracy of the subsequent judgment of the driver state based on the prediction result output by the neural network, identify the behavior of the driver that is not conducive to the safe driving of the vehicle in time, and greatly reduce the probability of accidents of the vehicle.

[0140] The driver state judgment method corresponding to the above embodiment, Figure 5 A structural schematic diagram of a driver state judgment device provided by the embodiment of the present application is shown. For ease of illustration, only parts related to the embodiment of the present application are shown. Figure 5 A structural schematic diagram of a driver state judgment device provided by the embodiment of the present application is shown. For ease of illustration, only parts related to the embodiment of the present application are shown.

[0141] The function determining module 501 is configured to obtain a true value and a predicted value of a regression task in a driver monitoring task, and determine a first continuous probability distribution function according to the true value and a second continuous probability distribution function according to the predicted value.

[0142] The function obtaining module 502 is configured to obtain a loss function corresponding to the regression task based on a loss function corresponding to a classification task in the driver monitoring task, and the first continuous probability distribution function and the second continuous probability distribution function.

[0143] The model obtaining module 503 is configured to obtain a trained neural network model based on a loss function corresponding to the regression task, wherein the neural network model inputs the first driver data and outputs the predicted value and the confidence parameter of the first driver.

[0144] The state determining module 504 is configured to obtain second driver data, obtain a predicted value and a confidence parameter of the second driver based on the second driver data and the neural network model, determine a confidence degree based on the confidence parameter of the second driver, and determine the state of the second driver according to the confidence degree and the predicted value of the second driver.

[0145] In a possible implementation, the function obtaining module 502 is specifically configured to:

[0146] obtain a first loss function based on a loss function corresponding to the classification task, and the first continuous probability distribution function and the second continuous probability distribution function;

[0147] determine a second loss function according to the first loss function and a true value of the regression task;

[0148] parameterize the second continuous probability distribution function, and obtain the loss function corresponding to the regression task based on the parameterized second continuous probability distribution function and the second loss function.

[0149] In a possible implementation, the function obtaining module 502 is specifically configured to:

[0150] parameterize the second continuous probability distribution function based on a Laplace distribution or a Gaussian distribution.

[0151] In a possible implementation, the model obtaining module 503 is specifically configured to:

[0152] set the loss function corresponding to the regression task as a loss function of a preset initial neural network model;

[0153] input the first driver data into the preset initial neural network model, and calculate a value of the loss function of the preset initial neural network model;

[0154] adjust the preset initial neural network model according to the value of the loss function, so that a value of the loss function of the adjusted preset initial neural network model satisfies a preset condition;

[0155] obtain the trained neural network model based on the adjusted preset initial neural network model,

[0156] In a possible implementation, the model obtaining module 503 is specifically configured to:

[0157] According to the preset neural network model convergence method, the loss function corresponding to the regression task is processed to obtain a third loss function;

[0158] Each term in the third loss function is respectively set with a loss weight to obtain a fourth loss function;

[0159] The fourth loss function is set as the loss function of the preset initial neural network model.

[0160] In a possible implementation, the state judging module 504 is specifically configured to:

[0161] It is judged whether the confidence degree is greater than a preset threshold, and the preset threshold is determined according to a minimum value of the confidence degree corresponding to the predicted value of the driver output by the neural network used for judging the state of the driver;

[0162] If the confidence degree is greater than the preset threshold, the state of the second driver is judged according to the predicted value of the second driver.

[0163] In a possible implementation, the confidence parameter of the second driver is determined according to the standard deviation of the neural network model;

[0164] The state judging module 504 is specifically configured to:

[0165] The confidence parameter of the second driver is processed by nonlinear mapping to obtain the confidence degree.

[0166] In a possible implementation, the first continuous probability distribution function is a Dirichlet function.

[0167] The apparatus provided by the embodiments of the present application can be used to execute the technical solutions of the above method embodiments, and the implementation principles and technical effects are similar, and the embodiments of the present application will not be repeated here.

[0168] Optionally, Figure 6 A possible basic hardware architecture schematic diagram of the driver monitoring system described in the present application is schematically provided.

[0169] Referring to Figure 6 , the driver monitoring system includes at least one processor 601 and a communication interface 603. Further optionally, it can also include a memory 602 and a bus 604.

[0170] Among the driver monitoring system, the number of processors 601 can be one or more, Figure 6Only one processor 601 is shown. The processor 601 can be a central processing unit (CPU), a graphics processing unit (GPU), or a digital signal processor (DSP). If the driver monitoring system has multiple processors 601, the multiple processors 601 can be of different types, or can be of the same type. Optionally, the multiple processors 601 of the driver monitoring system can also be integrated into a multi-core processor.

[0171] The memory 602 stores computer instructions and data; the memory 602 can store computer instructions and data required to implement the above-mentioned driver state determination method provided by the present application, for example, the memory 602 stores instructions for implementing the steps of the above-mentioned driver state determination method. The memory 602 can be any one or any combination of the following storage media: non-volatile memory (such as read-only memory (ROM), solid state disk (SSD), hard disk (HDD), optical disk), volatile memory.

[0172] The communication interface 603 can provide information input / output for the at least one processor. It can also include any one or any combination of the following devices: network interface (such as Ethernet interface), wireless network card, and other devices with network access function.

[0173] Optionally, the communication interface 603 can also be used for data communication between the driver monitoring system and other computing devices or terminals.

[0174] Further optionally, Figure 6 The bus 604 is represented by a thick line. The bus 604 can connect the processor 601 with the memory 602 and the communication interface 603. In this way, the processor 601 can access the memory 602 through the bus 604, and also use the communication interface 603 to interact with other computing devices or terminals.

[0175] In the present application, the driver monitoring system executes computer instructions in the memory 602, so that the driver monitoring system implements the above-mentioned driver state determination method provided by the present application, or so that the driver monitoring system deploys the above-mentioned driver state determination device.

[0176] From the perspective of logical function division, an example is as follows: Figure 6As shown, the memory 602 can include a function determining module 501, a function obtaining module 502, a model obtaining module 503, and a state determining module 504. The inclusion herein only relates to the functions of the function determining module, the function obtaining module, the model obtaining module, and the state determining module that can be respectively implemented when the instructions stored in the memory are executed, and is not limited to a physical structure.

[0177] In addition, the driver monitoring system described above can be used not only for the purpose of monitoring the driver as described above ​ The software implementation can also be implemented as a hardware module or a circuit unit through hardware implementation.

[0178] The present application provides a computer readable storage medium, the computer program product includes computer instructions, the computer instructions instruct the computing device to execute the driver state determination method provided by the present application.

[0179] The present application provides a computer program product, including computer instructions, the computer instructions are executed by the processor to execute the driver state determination method.

[0180] The present application provides a chip, including at least one processor and a communication interface, the communication interface provides information input and / or output for the at least one processor. Further, the chip can also contain at least one memory, the memory is used to store computer instructions. The at least one processor is used to call and run the computer instructions, so as to execute the driver state determination method provided by the present application.

[0181] In several embodiments provided by the present application, it should be understood that the disclosed apparatus and method can be implemented by other means. For example, the apparatus embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, apparatus or unit, and can be electrical, mechanical or other forms.

[0182] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0183] In addition, the various functional units in the embodiments of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of hardware plus software function units.

Claims

1. A method for determining driver status, characterized in that, include: Obtain the true value and predicted value of the regression task in the driver monitoring task, and determine the first continuous probability distribution function based on the true value and the second continuous probability distribution function based on the predicted value; Based on the loss function corresponding to the classification task in the driver monitoring task, as well as the first continuous probability distribution function and the second continuous probability distribution function, the loss function corresponding to the regression task is obtained; Based on the loss function corresponding to the regression task, a trained neural network model is obtained, wherein the neural network model takes the first driver data as input and outputs the predicted value and confidence parameter of the first driver. Second driver data is acquired. Based on the second driver data and the neural network model, the predicted value and confidence parameter of the second driver are obtained. Based on the confidence parameter of the second driver, the confidence level is determined. Based on the confidence level and the predicted value of the second driver, the state of the second driver is judged. The first driver data and the first driver data are both facial images of the driver. The step of determining the state of the second driver based on the confidence level and the predicted value of the second driver includes: Determine whether the confidence level is greater than a preset threshold, wherein the preset threshold is determined based on the minimum confidence level corresponding to the driver's predicted value output by the neural network used to determine the driver's state; If the confidence level is greater than the preset threshold, the state of the second driver is judged based on the predicted value of the second driver.

2. The method according to claim 1, characterized in that, The step of obtaining the loss function corresponding to the regression task based on the loss function corresponding to the classification task in the driver monitoring task, and the first continuous probability distribution function and the second continuous probability distribution function, includes: Based on the loss function corresponding to the classification task, and the first continuous probability distribution function and the second continuous probability distribution function, a first loss function is obtained; Based on the first loss function and the true value of the regression task, determine the second loss function; The second continuous probability distribution function is parameterized, and the loss function corresponding to the regression task is obtained based on the parameterized second continuous probability distribution function and the second loss function.

3. The method according to claim 2, characterized in that, The parameterization of the second continuous probability distribution function includes: The second continuous probability distribution function is parameterized based on the Laplace distribution or Gaussian distribution.

4. The method according to any one of claims 1 to 3, characterized in that, The process of obtaining the trained neural network model based on the loss function corresponding to the regression task includes: Set the loss function corresponding to the regression task to the loss function of the preset initial neural network model; The first driver data is input into the preset initial neural network model, and the value of the loss function of the preset initial neural network model is calculated; Based on the value of the loss function, the preset initial neural network model is adjusted so that the value of the loss function of the adjusted preset initial neural network model meets the preset conditions; Based on the adjusted preset initial neural network model, the trained neural network model is obtained.

5. The method according to claim 4, characterized in that, Before setting the loss function corresponding to the regression task as the loss function of the preset initial neural network model, the method further includes: Based on the preset neural network model convergence method, the loss function corresponding to the regression task is processed to obtain a third loss function; A loss weight is assigned to each term in the third loss function to obtain the fourth loss function; Setting the loss function corresponding to the regression task as the loss function of the preset initial neural network model includes: The fourth loss function is set as the loss function of the preset initial neural network model.

6. The method according to any one of claims 1 to 3, characterized in that, The confidence parameter of the second driver is determined based on the standard deviation of the neural network model; Determining the confidence level based on the confidence parameters of the second driver includes: The confidence level is obtained by performing nonlinear mapping processing on the confidence parameters of the second driver.

7. The method according to any one of claims 1 to 3, characterized in that, The first continuous probability distribution function is the Dirichlet function.

8. A driver status determination device, characterized in that, include: The function determination module is used to obtain the true value and predicted value of the regression task in the driver monitoring task, and determine the first continuous probability distribution function based on the true value and the second continuous probability distribution function based on the predicted value. The function acquisition module is used to obtain the loss function corresponding to the regression task based on the loss function corresponding to the classification task in the driver monitoring task, as well as the first continuous probability distribution function and the second continuous probability distribution function; The model acquisition module is used to obtain a trained neural network model based on the loss function corresponding to the regression task, wherein the neural network model takes the first driver data as input and outputs the predicted value and confidence parameter of the first driver. The state determination module is used to acquire second driver data, obtain the predicted value and confidence parameter of the second driver based on the second driver data and the neural network model, determine the confidence level based on the confidence parameter of the second driver, and determine the state of the second driver based on the confidence level and the predicted value of the second driver; the first driver data and the first driver data are both facial images of the driver; The state determination module is specifically used to determine whether the confidence level is greater than a preset threshold. The preset threshold is determined based on the minimum confidence level corresponding to the driver's predicted value output by the neural network used to determine the driver's state. If the confidence level is greater than the preset threshold, the state of the second driver is determined based on the predicted value of the second driver.

9. A driver monitoring system, characterized in that, include: processor; Memory; as well as Computer programs; The computer program is stored in the memory and configured to be executed by the processor, the computer program including instructions for performing the method as described in any one of claims 1-7.

Images