Method and system of analyzing an abnormal event for a driver monitoring system

Abstract

A system of analyzing an abnormal event is used to analyze an abnormal event determined by an in-car image, and includes a processing unit that is configured to perform the following operations: using multiple characteristic parameters to analyze multiple failed in-car image data and multiple valid in-car image data in order to establish an event type classifier; using the event type classifier to obtain an event type of the abnormal event according to the in-car image; determining whether the event type of the abnormal event is one of the N failure types; and generating a stop upload request when the event type of the abnormal event is determined to be one of the N failure types.

Description

Technical Field

[0001] This invention relates to an abnormal event analysis method and system, and more particularly to an abnormal event analysis method and system for a driver monitoring system. Prior Technology

[0002] Currently, fleet management companies can use connected dashcams with in-vehicle camera modules to monitor driver mental state and identify any abnormal events such as fatigued driving or distracted driving. This connected dashcam detects abnormal events based on driver monitoring footage captured by the in-vehicle camera module and transmits the relevant image data for each detected abnormal event to a remote server via an external mobile network for further analysis.

[0003] However, if the aforementioned networked dashcams frequently fail due to undetected or non-human factors (such as the installation and positioning of the image capture module, weather conditions, etc.), transmitting a large amount of invalid (failed) video data of these abnormal events will result in wasted network transmission costs and will also hinder the accuracy of subsequent analysis of the driver's driving behavior.

[0004] Therefore, for abnormal events detected by the Driver Monitoring System (DMS), how to invent an abnormal event analysis method that can reduce the waste of network transmission resources and quickly and accurately analyze the effectiveness or failure cause of abnormal events has become one of the issues that related technical fields want to solve. Summary of the Invention

[0005] Therefore, the purpose of this invention is to provide an abnormal event analysis method and system that can overcome at least one drawback of the prior art.

[0006] Therefore, the present invention provides an abnormal event analysis method for analyzing an abnormal event related to the driver's mental or concentration state, as determined by a driver monitoring system based on an in-vehicle image of a driver. This method is executed using a computer system and includes the following steps: (A) collecting information about N... The N types of failures include at least one type indicating that the location of the imaging device used by the driver monitoring system to capture the in-vehicle image is obstructed by other objects, another type indicating that the ambient light is too dark when the imaging device captures the in-vehicle image, and yet another type indicating that the lens of the imaging device is positioned at the wrong angle; (B) the failures and the valid in-vehicle images are analyzed using at least the image gradient change rate, the face recognition probability, and the average image brightness value as multiple feature parameters. (C) Based on the in-vehicle image, using the event type classifier, obtain an event type representing the abnormal event, which is one of the N failure types and the valid type; (D) Determine whether the event type of the abnormal event is one of the N failure types; and (E) When it is determined that the event type of the abnormal event is one of the N failure types, generate a stop upload request to stop the driver monitoring system from uploading the in-vehicle image to a server, and transmit it to the driver monitoring system.

[0007] Therefore, the present invention provides an abnormal event analysis system for analyzing an abnormal event related to the driver's mental or concentration state determined by a driver monitoring system based on an in-vehicle image of a driver, and includes a receiving module, a storage unit and a processing unit.

[0008] The receiving module is adapted to connect to the driver monitoring system to receive the in-vehicle image corresponding to the abnormal event.

[0009] This storage unit stores information about N collected in advance. The N types of failure data include multiple failed in-vehicle video records of different failure types and multiple valid in-vehicle video records of a valid type. The N types of failure types include at least one type indicating that the location of the imaging device of the driver monitoring system used to capture the in-vehicle video is obstructed by other objects, one type indicating that the ambient brightness is too low when the imaging device captures the in-vehicle video, and one type indicating that the lens of the imaging device is positioned at the wrong angle.

[0010] The processing unit is electrically connected to the receiving module and the storage unit. It receives the in-vehicle images received by the receiving module and analyzes the failed and valid in-vehicle image data stored in the storage unit using at least image gradient change rate, face recognition probability, and average image brightness value as multiple feature parameters. This analysis establishes a multi-dimensional event type classifier for the N types of failures and the valid type. Based on the in-vehicle images, the event type classifier obtains an event type representing the abnormal event, which is one of the N types of failures and the valid type. It determines whether the event type of the abnormal event is one of the N types of failures. When it is determined that the event type of the abnormal event is one of the N types of failures, a stop upload request is generated to prevent the driver monitoring system from uploading the in-vehicle images to a server.

[0011] The advantage of this invention lies in its ability to re-verify the validity of events already identified as abnormal by the driver monitoring system using an event type classifier. If the event type obtained by the event type classifier corresponds to a valid type, the abnormal event is valid; if the obtained event type corresponds to one of the N failure types, the abnormal event is invalid, and the possible cause of the failure can be determined by the event type. Furthermore, knowing that the abnormal event is invalid facilitates subsequent adjustments to the driver monitoring system, such as generating a stop upload request to prevent the driver monitoring system from uploading the in-vehicle images to the server, thereby reducing the waste of network transmission resources. Simple Explanation of the Diagram

[0012] Other features and effects of the present invention will be clearly presented in the embodiments with reference to the drawings, wherein: Figure 1 is a block diagram illustrating, by way of example, an abnormal event analysis system according to an embodiment of the present invention, and a driver monitoring system used in conjunction with it; Figure 2 is a flowchart illustrating, by way of example, how a processing unit of this embodiment executes a modeling procedure; and Figure 3 is a flowchart illustrating, by way of example, how the processing unit of this embodiment executes an exception analysis procedure. Implementation

[0013] Before the invention is described in detail, it should be noted that similar elements are represented by the same numbers in the following description.

[0014] Referring to Figure 1, an abnormal event analysis system 1 according to an embodiment of the present invention is used to analyze an abnormal event related to the driver's mental or concentration state, determined by a driver monitoring system 2 based on an in-vehicle image of a driver. For example, the abnormal event may be an event such as fatigued driving or distracted driving that occurs during driving. The abnormal event analysis system 1 can be implemented as, for example, but not limited to, a computer system, and may include, for example, a receiving module 11, a storage unit 12, a processing unit 13, and an output module 14.

[0015] The receiving module 11 is adapted to connect to the driver monitoring system 2 to receive the in-vehicle image corresponding to the abnormal event.

[0016] The storage unit 12 stores information about N collected in advance. The system contains multiple failed in-vehicle image data sets of different failure types, and multiple valid in-vehicle image data sets of one valid type. In this embodiment, the N failure types are at least related to the location of the image capturing device included in the driver monitoring system 2 for capturing the in-vehicle images, the lens positioning of the image capturing device, and the ambient brightness when the image capturing device captures the in-vehicle images.

[0017] For example, the storage unit 12 stores multiple failed in-vehicle image data related to three (i.e., N=3) different failure types, and multiple valid in-vehicle image data related to a valid type D indicating distracted driving. The three failure types are, respectively, failure type A, indicating that the installation position of the imaging device is obstructed by other objects; failure type B, indicating that the ambient brightness is too low when the imaging device captures the in-vehicle image; and failure type C, indicating that the lens of the imaging device is positioned at an incorrect angle.

[0018] The processing unit 13 is electrically connected to the receiving module 11 and the storage unit 12, and receives the in-vehicle image received by the receiving module 11. The detailed operation of the processing unit 13 and its components will be explained below.

[0019] The output module 14 is electrically connected to the driver monitoring system 2 and the processing unit 13 and is controlled by the processing unit 13.

[0020] Before use, the processing unit 13 must first execute a modeling procedure to establish a multi-dimensional event type classifier for the N types of failures and the effective type. The following will describe in detail how the processing unit 13 executes the modeling procedure with reference to Figures 1 and 2. The modeling procedure includes steps S21 to S24.

[0021] In step S21, the processing unit 13 calculates a set of corresponding feature parameter values ​​based on each of the failed in-vehicle image data and the valid in-vehicle image data stored in the storage unit 12. In this embodiment, the feature parameter values ​​include an image gradient change rate, a face recognition probability, and an image average brightness value. The image gradient change rate indicates the average rate of change of grayscale values ​​between adjacent pixels after binarization; the face recognition probability indicates the probability that the image represents a face; and the image average brightness value is the average of the brightness values ​​of all pixels in the image.

[0022] It is worth mentioning that in other implementations, the feature parameter value also includes, for example, an angle value indicating the bending angle of the driver's arm when holding the steering wheel, which can be obtained by existing skeletal recognition technology.

[0023] In step S22, the processing unit 13 calculates all feature parameter values ​​corresponding to the failed in-vehicle image data and the valid in-vehicle image data, and performs statistics on each feature parameter to obtain the Gaussian probability distribution for each of the N failure types and the valid type. Furthermore, the N failure types and the valid type can be distinguished based on the Gaussian probability distribution of these feature parameters for different types.

[0024] For example, based on the in-vehicle image data of failure type A, failure type B and failure type C in the aforementioned examples, and the in-vehicle image data of effective type D, the Gaussian probability distribution of these characteristic parameters is statistically shown in Table 1 below. Table 1 Event Type Image gradient change rate facial recognition probability Image average brightness value Failure type A Small Small Too big Failure type B medium medium Small Failure type C Too big Small Too big Valid type D Too big Too big Too big

[0025] In step S23, the processing unit 13 constructs a multi-dimensional matrix using these feature parameters, and represents all the obtained Gaussian probability distributions as a mean matrix and a covariance matrix.

[0026] Finally, in step S24, the processing unit 13 uses a multivariate Gaussian distribution model to establish the event type classifier based on the mean matrix and the covariance matrix. At this point, the modeling process is complete.

[0027] In use, when the processing unit 13 receives the in-vehicle image received by the receiving module 11 from the driver monitoring system 2, the processing unit 13 executes an abnormal event analysis procedure to obtain an event type of the abnormal event. The flow of the abnormal event analysis procedure will be described in detail below with reference to Figures 1 and 3. The abnormal event analysis procedure includes steps S31 to S36.

[0028] In step S31, the processing unit 13 calculates an image gradient change rate, a face recognition probability, and an average image brightness value corresponding to the abnormal event based on the in-vehicle image.

[0029] In step S32, the processing unit 13 uses the established event type classifier to estimate the probability value (i.e., event probability value) of the abnormal event being classified into each of the N failure types and the valid type based on the calculated image gradient change rate, face recognition probability, and image brightness value.

[0030] In step S33, the processing unit 13 selects one of the N failure types and the effective type that corresponds to the estimated maximum probability value as the event type representing the abnormal event.

[0031] In step S34, the processing unit 13 determines whether the event type of the abnormal event is one of the N failure types. When it is determined that the event type of the abnormal event is one of the N failure types, the process proceeds to step S35; when it is determined that the event type of the abnormal event is not one of the N failure types, the abnormal event is valid, and the process proceeds to step S36.

[0032] In step S35, the processing unit 13 generates a stop upload request to stop the driver monitoring system 2 from uploading the in-vehicle image to a server (not shown), and outputs the stop upload request to the driver monitoring system 2 via the output module 14.

[0033] In step S36, the processing unit 13 generates an abnormal event valid notification and outputs the abnormal event valid notification to the driver monitoring system 2 via the output module 14, so that the driver monitoring system 2 can normally upload the in-vehicle image to the server.

[0034] In summary, by using the event type classifier to re-verify events already identified as abnormal by the driver monitoring system 2, the validity of the abnormal events is confirmed. If the event type obtained by the event type classifier corresponds to the valid type, the abnormal event is valid; if the obtained event type corresponds to one of the N failure types, the abnormal event is invalid, and the possible cause of the failure can be determined by the event type. Furthermore, knowing that the abnormal event is invalid facilitates subsequent adjustments to the driver monitoring system 2, such as generating a stop upload request to stop the driver monitoring system 2 from uploading the in-vehicle images to the server, thereby reducing the waste of network transmission resources. Therefore, the purpose of this invention is indeed achieved.

[0035] However, the above description is merely an embodiment of the present invention and should not be construed as limiting the scope of the present invention. Any simple equivalent changes and modifications made in accordance with the scope of the patent application and the contents of the patent specification shall still fall within the scope of the patent of the present invention.

[0036] 1: Abnormal Event Analysis System 11: Receiver Module 12: Storage Unit 13: Processing Unit 14: Output Module 2: Driver monitoring system S21~S24: Steps S31~S36: Steps

Claims

1. An abnormal event analysis method for analyzing an abnormal event related to the driver's mental or concentration state determined by a driver monitoring system based on an in-vehicle image of a driver, executed using a computer system, the abnormal event failure analysis method comprising the following steps: (A) collecting multiple failed in-vehicle image data for N different failure types and multiple valid in-vehicle image data for a valid type, wherein the N failure types include at least one failure type indicating that the location of an image capturing device included in the driver monitoring system for capturing the in-vehicle image is obstructed by other objects, a failure type indicating that the ambient brightness is too low when the image capturing device captures the in-vehicle image, and a failure type indicating that the lens of the image capturing device is positioned at an incorrect angle; (B) analyzing the failed in-vehicle image data and the valid in-vehicle image data using at least image gradient change rate, face recognition probability, and average image brightness value as multiple feature parameters to establish a multi-dimensional event type classifier for the N failure types and the valid type; (C) Based on the in-vehicle image, using the event type classifier, obtain an event type representing the abnormal event, the event type being one of the N failure types and the valid type; (D) Determine whether the event type of the abnormal event is one of the N failure types; and (E) When it is determined that the event type of the abnormal event is one of the N failure types, generate a stop upload request to stop the driver monitoring system from uploading the in-vehicle image to a server, and transmit it to the driver monitoring system.

2. The abnormal event analysis method as described in request item 1, wherein, Step (B) includes the following sub-steps: (B1) For each of the failed in-vehicle image data and the valid in-vehicle image data, calculate a set of feature parameter values ​​that include at least one image gradient change rate, one face recognition probability, and one image average brightness value; (B2) For all the feature parameter values ​​calculated for the failed in-vehicle image data and the valid in-vehicle image data respectively, perform statistics on each feature parameter to obtain the Gaussian probability distribution for each of the N failure types and the valid type; (B3) Construct a multi-dimensional matrix with the feature parameters, and represent all the obtained Gaussian probability distributions as a mean matrix and a covariance matrix; and (B4) Build an event type classifier based on the mean matrix and the covariance matrix using a multivariate Gaussian distribution model.

3. The abnormal event analysis method as described in request item 1, wherein, Step (C) includes the following sub-steps: (C1) Calculate an image gradient change rate, a face recognition probability, and an image average brightness value corresponding to the abnormal event based on the in-vehicle image; (C2) Using the event type classifier, estimate the probability value of the abnormal event being classified into each of the N failure types and the effective type based on the calculated image gradient change rate, the face recognition probability, and the image average brightness value; and (C3) Select one of the N failure types and the effective type that corresponds to the estimated maximum probability value as the event type representing the abnormal event.

4. An abnormal event analysis system for analyzing an abnormal event related to the driver's mental or concentration state determined by a driver monitoring system based on an in-vehicle image of a driver, and comprising: a receiving module adapted to connect to the driver monitoring system to receive the in-vehicle image corresponding to the abnormal event; a storage unit storing pre-collected data on multiple failed in-vehicle images of N different failure types and multiple valid in-vehicle images of a valid type, wherein the N failure types include at least one failure type indicating that the location of an image capturing device included in the driver monitoring system for capturing the in-vehicle image is obstructed by other objects, one failure type indicating that the ambient brightness is too low when the image capturing device captures the in-vehicle image, and one failure type indicating that the lens of the image capturing device is positioned at an incorrect angle; and a processing unit electrically connected to the receiving module and the storage unit, receiving the in-vehicle image received by the receiving module and assembling it for: The system analyzes the failed and valid in-vehicle image data stored in the storage unit using at least several feature parameters, including image gradient change rate, face recognition probability, and average image brightness value, to establish a multi-dimensional event type classifier for the N types of failures and the valid type. Based on the in-vehicle image, the event type classifier is used to obtain an event type representing the abnormal event, which is one of the N types of failures and the valid type. The system determines whether the event type of the abnormal event is one of the N types of failures, and when it is determined that the event type of the abnormal event is one of the N types of failures, a stop upload request is generated to stop the driver monitoring system from uploading the in-vehicle image to a server.

5. The abnormal event analysis system as described in claim 4 further includes: an output module connected to and controlled by the driver monitoring system and the processing unit to output the stop upload request to the driver monitoring system.

6. The exception event analysis system as described in claim 4, wherein, The processing unit establishes the event type classifier through the following operations: Based on each of the failed in-vehicle image data and the valid in-vehicle image data stored in the storage unit, a set of feature parameter values ​​corresponding to at least one image gradient change rate, one face recognition probability, and one image average brightness value are calculated; For all the feature parameter values ​​calculated corresponding to the failed in-vehicle image data and the valid in-vehicle image data respectively, statistics are performed on each feature parameter to obtain the Gaussian probability distribution for each of the N failure types and the valid type; A multi-dimensional matrix is ​​constructed using these feature parameters, and all the obtained Gaussian probability distributions are represented as an average matrix and a covariance matrix; and the event type classifier is established based on the average matrix and the covariance matrix using a multivariate Gaussian distribution model.

7. The exception event analysis system as described in claim 4, wherein, The processing unit obtains the event type of the abnormal event through the following operations: Based on the in-vehicle image, it calculates an image gradient change rate, a face recognition probability, and an image average brightness value corresponding to the abnormal event; using the established event type classifier, it estimates the probability value of classifying the abnormal event into each of the N failure types and the effective type based on the calculated image gradient change rate, the face recognition probability, and the image average brightness value; and selects the type with the estimated maximum probability value among the N failure types and the effective type as the event type representing the abnormal event.

8. An abnormal event analysis method for analyzing an abnormal event related to the driver's mental or concentration state determined by a driver monitoring system based on an in-vehicle image of a driver, executed using a computer system, the abnormal event failure analysis method comprising the following steps: (A) collecting multiple failed in-vehicle image data and multiple valid in-vehicle image data; (B) analyzing the failed in-vehicle image data and the valid in-vehicle image data with multiple feature parameters to establish an event type classifier; (C) using the event type classifier, and combining the type of the maximum probability value estimated by the event probability value as an event type representing the abnormal event, the event type being one of multiple failure types and a valid type; and (D) when the event type of the abnormal event is one of the failure types, generating a stop upload request to stop uploading the in-vehicle image to a server.

Images