Fatigue driving detection system

Abstract

The embodiment of the invention discloses a fatigue driving detection system, and the system comprises an object information collection module which is used for collecting image information and sound information of a driving object driving a target vehicle in the driving process of the target vehicle, transmitting the image information to an image processing module, and transmitting the sound information to a sound processing module; the image processing module is used for performing feature extraction on the received image information to obtain facial features of the driving object; the sound processing module is used for performing feature extraction on the received sound information to obtain sound features of the driving object; and the fatigue detection module is used for acquiring the facial features from the image processing module, acquiring the sound features from the sound processing module, and performing fatigue detection on the driving object based on the facial features and the sound features to obtain a detection result. According to the technical scheme provided by the embodiment of the invention, the fatigue driving of the driving object can be accurately detected.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of data processing, and in particular, to a fatigue driving detection system. Background Art

[0002] In recent years, with the rapid development of transportation, vehicles have become the main choice for long-distance travel. After a long time of driving, it will cause driver fatigue, thus increasing the occurrence of traffic accidents.

[0003] However, the existing fatigue driving detection solutions have low accuracy and urgently need to be solved. Summary of the Invention

[0004] Embodiments of the present invention provide a fatigue driving detection system to achieve accurate detection of fatigue driving of a driving object.

[0005] According to one aspect of the present invention, a fatigue driving detection system is provided, which may include: an object information acquisition module, an image processing module, a sound processing module, and a fatigue detection module; wherein,

[0006] The object information acquisition module is configured to collect image information and sound information of a driving object driving a target vehicle during the driving process of the target vehicle, send the image information to the image processing module, and send the sound information to the sound processing module;

[0007] The image processing module is configured to extract features from the received image information to obtain the facial features of the driving object;

[0008] The sound processing module is configured to extract features from the received sound information to obtain the sound features of the driving object;

[0009] The fatigue detection module is configured to obtain the facial features from the image processing module, and obtain the sound features from the sound processing module, and perform fatigue detection on the driving object based on the facial features and the sound features to obtain a detection result.

[0010] The technical solution of the embodiment of the present invention includes an object information acquisition module, an image processing module, a sound processing module, and a fatigue detection module. Among them, the object information acquisition module is used to collect the image information and sound information of the driving object driving the target vehicle during the driving process of the target vehicle, and send the image information to the image processing module, and send the sound information to the sound processing module to achieve all-round acquisition of the driving information of the driving object. The image processing module is used to extract features from the received image information to obtain the facial features of the driving object, so as to accurately detect the fatigue of the driving object based on the facial features in the follow-up. The sound processing module is used to extract features from the received sound information to obtain the sound features of the driving object, so as to accurately detect the fatigue of the driving object based on the sound features in the follow-up. The fatigue detection module is used to obtain the facial features from the image processing module and obtain the sound features from the sound processing module, and perform fatigue detection on the driving object based on the facial features and the sound features to obtain a detection result, so as to jointly perform fatigue detection on the driving object by combining the facial features and the sound features, and improve the accuracy of fatigue detection. The above technical solution combines the facial features and the sound features to jointly perform fatigue detection on the driving object, and can achieve accurate detection of the fatigue driving of the driving object.

[0011] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0013] Figure 1 is a structural block diagram of a fatigue driving detection system provided according to an embodiment of the present invention;

[0014] Figure 2 is a structural block diagram of another fatigue driving detection system provided according to an embodiment of the present invention;

[0015] Figure 3 is a structural block diagram of yet another fatigue driving detection system provided according to an embodiment of the present invention;

[0016] Figure 4 is a fatigue driving detection flow chart of a specific example in yet another fatigue driving detection system provided according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0018] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. The situations of "target", "original", etc. are similar and will not be elaborated here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0019] Figure 1 is a structural block diagram of a fatigue driving detection system provided by an embodiment of the present invention. This embodiment is applicable to the situation of detecting fatigue driving of a driving object of a driving vehicle.

[0020] See Figure 1 , the fatigue driving detection system of the embodiment of the present invention includes: an object information acquisition module 11, an image processing module 12, a sound processing module 13, and a fatigue detection module 14; where

[0021] The object information acquisition module 11 is configured to collect image information and sound information of a driving object driving a target vehicle during the driving process of the target vehicle, and send the image information to the image processing module 12, and send the sound information to the sound processing module 13;

[0022] The image processing module 12 is configured to extract features from the received image information to obtain the facial features of the driving object;

[0023] The sound processing module 13 is configured to extract features from the received sound information to obtain the sound features of the driving object;

[0024] The fatigue detection module 14 is used to obtain facial features from the image processing module 12 and voice features from the voice processing module 13, and perform fatigue detection on the driving object based on the facial features and voice features to obtain a detection result.

[0025] Among them, the object information acquisition module 11 can be understood as a module for acquiring image information and voice information of the driving object of the target vehicle during driving. Optionally, a video acquisition device, such as a high-definition camera, can be used to capture the facial image of the driver to ensure stable operation in complex environments such as light changes and wearing glasses. Optionally, the high-definition camera can be installed at a suitable position inside the cab to minimize interference with the driving line of sight of the driving object; a voice acquisition device, such as a high-sensitivity microphone, can be used to acquire the voice information of the driving object (such as sounds of yawning, snoring, and coughing). Optionally, multiple microphones can be installed around the driving object to acquire the voice information of the driving object in all directions.

[0026] The target vehicle can be understood as a vehicle capable of performing fatigue driving detection on the driving object of the target vehicle during driving.

[0027] The image processing module 12 can be understood as a module for processing the acquired image information, which can process the received image information and extract facial features from the image information. Optionally, the image processing module 12 can use a trained facial feature extraction model to extract features from the received image information, such as extracting facial features such as the blinking frequency and eyelid closing degree of the driving object and head movement features (such as nodding and shaking the head).

[0028] The voice processing module 13 can be understood as a module for extracting voice features from the voice information of the driving object. Optionally, the voice information can be preprocessed first, such as filtering and noise reduction, and then a trained voice feature extraction model can be used to extract features from the received voice information to obtain voice features.

[0029] The detection result can be understood as a judgment result on whether the driving object is fatigued while driving. Optionally, in order to better determine the state of the driving object, the detection result can be further divided into multiple detection levels, such as non-fatigue driving, mild fatigue driving, moderate fatigue driving, and severe fatigue driving, etc.

[0030] The fatigue detection module 14 can be understood as a module for detecting the fatigue level of the driving object. It can perform fatigue detection on the driving object based on the facial features obtained from the image processing module 12 and the voice features obtained from the voice processing module 13 to obtain a detection result. Optionally, a large number of image features and voice features can be used to train a neural network model so that the neural network model can identify fatigue features and detect the fatigue level, thereby obtaining a fatigue detection model. The fatigue detection module 14 can use the fatigue detection model to perform fatigue detection on the driving object to obtain a detection result.

[0031] Through the object information acquisition module 11, during the driving process of the target vehicle, the image information and voice information of the driving object are collected, and the image information is sent to the image processing module 12, and the voice information is sent to the voice processing module 13; the image processing module 12 performs feature extraction on the received image information to obtain the facial features of the driving object; the voice processing module 13 performs feature extraction on the received voice information to obtain the voice features of the driving object; the fatigue detection module 14 performs fatigue detection on the driving object based on the obtained facial features and voice features, comprehensively analyzes the all-round behavior of the driving object, and thus obtains a detection result.

[0032] The technical solution of the embodiment of the present invention includes an object information acquisition module, an image processing module, a voice processing module, and a fatigue detection module; wherein, the object information acquisition module is used to collect the image information and voice information of the driving object driving the target vehicle during the driving process of the target vehicle, and send the image information to the image processing module, and send the voice information to the voice processing module to achieve the all-round acquisition of the driving information of the driving object; the image processing module is used to perform feature extraction on the received image information to obtain the facial features of the driving object, so as to perform accurate fatigue detection on the driving object based on the facial features subsequently; the voice processing module is used to perform feature extraction on the received voice information to obtain the voice features of the driving object, so as to perform accurate fatigue detection on the driving object based on the voice features subsequently; the fatigue detection module is used to obtain the facial features from the image processing module and the voice features from the voice processing module, and perform fatigue detection on the driving object based on the facial features and voice features to obtain a detection result, thereby combining the facial features and voice features to jointly perform fatigue detection on the driving object and improve the accuracy of fatigue detection. The above technical solution combines facial features and voice features to jointly perform fatigue detection on the driving object, and can achieve accurate detection of the driving object's fatigue driving.

[0033] An alternative technical solution is that multiple sound collection devices are arranged in the target vehicle. The object information collection module is further configured to: determine the sound source position of the sound source corresponding to the sound information based on the time when the collected sound information reaches each sound collection device and the installation positions of the multiple sound collection devices, and send the sound information to the sound processing module when the sound source position is within the preset driving position range of the target vehicle.

[0034] Among them, the sound collection device can be understood as a device for collecting the sound in the cab of the target vehicle, such as a microphone, a sound sampler, a pickup, etc. Multiple sound collection devices can be installed in the target vehicle to collect sound information. Optionally, in order to improve the accuracy of collecting the sound information of the driving object, the sound collection device can be installed around the driving position of the target vehicle, including the co-pilot, the driver's seat, the left and right rear seats, etc., to achieve multi-zone collection.

[0035] The installed sound collection devices can form a distributed array, so that the sound source position of the sound source corresponding to the sound information can be calculated based on the time when the collected sound information reaches each sound collection device and the installation positions of the multiple sound collection devices. When the sound source position is within the preset driving position range of the target vehicle, it can be determined that the sound information is emitted by the driving object, and thus the sound information is sent to the sound processing module.

[0036] The above technical solution can accurately collect the sound information of the driving object inside the target vehicle and avoid the interference of the sounds emitted by other passengers.

[0037] Another alternative technical solution further includes: a third start signal sending module; among them, the object information collection module is further configured to collect the grip force applied by the driving object on the steering wheel of the target vehicle and send the grip force to the third start signal sending module; the third start signal sending module is configured to send a third start signal to the fatigue detection module when the received grip force value is less than a preset grip force threshold; the fatigue detection module is further configured to obtain the facial features from the image processing module and the sound features from the sound processing module when receiving the third start signal, and perform fatigue detection on the driving object based on the facial features and the sound features to obtain a detection result.

[0038] Among them, the third start signal sending module can be understood as a module for judging whether to perform fatigue detection on the driving object and sending a third start signal for starting fatigue detection to the fatigue detection module.

[0039] The object information acquisition module can also collect the grip force exerted by the driving object on the steering wheel of the target vehicle and send the grip force to the third start signal sending module. Since the grip force of the driving object on the steering wheel will weaken at the moment of fatigue, the third start signal sending module can send a third start signal to the fatigue detection module when the received grip force value is less than the preset grip force threshold. When receiving the third start signal, the fatigue detection module obtains the facial features from the image processing module and the voice features from the voice processing module, and based on the facial features and the voice features, performs fatigue detection on the driving object to obtain the detection result.

[0040] The above technical solution can start fatigue detection in a timely manner by detecting the grip force, improving the timeliness of fatigue detection.

[0041] Figure 2 It is another structural block diagram provided by an embodiment of the present invention. This embodiment is optimized based on the above technical solutions. In this embodiment, optionally, the fatigue detection module includes a feature acquisition sub-module and a fatigue detection sub-module; wherein, the feature acquisition sub-module is used to obtain facial features from the image processing module and voice features from the voice processing module; the fatigue detection sub-module is used to determine the fatigue degree of the driving object based on the obtained facial features and voice features, and perform fatigue detection on the driving object based on the fatigue degree to obtain the detection result.

[0042] Among them, the explanations of the same or corresponding terms as those in the above embodiments are not repeated here.

[0043] Specifically, referring to Figure 2 , the fatigue driving detection system of this embodiment includes: an object information acquisition module 21, an image processing module 22, a voice processing module 23, and a fatigue detection module 24, wherein the fatigue detection module 24 includes a feature acquisition sub-module 241 and a fatigue detection sub-module 242; wherein,

[0044] The object information acquisition module 21 is used to collect the image information and voice information of the driving object driving the target vehicle during the driving process of the target vehicle, and send the image information to the image processing module 22 and the voice information to the voice processing module 23;

[0045] The image processing module 22 is used to extract features from the received image information to obtain the facial features of the driving object;

[0046] The voice processing module 23 is used to extract features from the received voice information to obtain the voice features of the driving object;

[0047] The feature acquisition sub-module 241 is used to obtain facial features from the image processing module 22 and voice features from the voice processing module 23;

[0048] The fatigue detection sub-module 242 is configured to determine the fatigue level of the driving object based on the acquired facial features and voice features, and perform fatigue detection on the driving object based on the fatigue level to obtain a detection result.

[0049] Among them, the facial features and voice features can reflect the fatigue level of the driving object. The number of blinks, the degree of eyelid closure, and the yawning frequency of the driving object can all reflect the fatigue level of the driving object. The more the number of blinks or the greater the yawning frequency, the more fatigued the driving object can be considered. Optionally, the fatigue level of the driving object can be determined in various ways, such as analyzing based on a neural network model, or judging based on multiple thresholds such as a preset blink threshold and a yawning threshold. Further, fatigue detection is performed on the driving object based on the fatigue level to obtain a detection result.

[0050] In the above technical solution, the fatigue level of the driving object is comprehensively determined through the acquired facial features and voice features, and the fatigue detection of the driving object can be accurately performed through the fatigue level.

[0051] An optional technical solution is that the fatigue detection sub-module includes a fatigue level determination unit and a fatigue driving judgment unit. Among them, the fatigue level determination unit is configured to input the acquired facial features and voice features into a pre-trained fatigue level determination model to obtain the fatigue level of the driving object. The fatigue driving judgment unit is configured to perform fatigue detection on the driving object according to whether the fatigue level is greater than a preset fatigue threshold to obtain a detection result.

[0052] Among them, the fatigue level determination model can be understood as a model obtained through training for determining the fatigue level of the driving object. Optionally, the fatigue level determination model can be obtained by training a neural network model with a training sample set composed of a large number of facial features and voice features.

[0053] Fatigue level determination unit

[0054] In the above technical solution, the fatigue level of the driving object can be accurately determined through the fatigue level determination model, so as to further obtain a more accurate detection result.

[0055] On this basis, optionally, it further includes: a data recording module, configured to acquire and upload image features, voice features, and detection results to the vehicle data center, so that the vehicle data center updates the preset fatigue threshold based on the image features, voice features, and detection results, and sends the updated preset fatigue threshold to the fatigue driving judgment unit.

[0056] Among them, the data recording module can be understood as a module that collects and uploads image features, sound features, and detection results. The data recording module can upload the obtained image features, sound features, and detection results to the vehicle data center. Based on the image features, sound features, and detection results, the vehicle data center verifies the detection results and can update the preset fatigue threshold according to the driving habits of the driving object. For example, if it is detected that the driving object has the habit of blinking frequently, the preset fatigue threshold will be increased. Then, the vehicle data center sends the updated preset fatigue threshold to the fatigue driving judgment unit.

[0057] In the above technical solution, by uploading the data of the driving object, the preset fatigue threshold can be adjusted personalized, improving the accuracy of fatigue detection.

[0058] Another alternative technical solution further includes: an early warning module, which is used to determine the fatigue level of the driving object based on the fatigue degree in the case where the detection result is fatigue driving, and execute the early warning measures corresponding to the fatigue level.

[0059] Among them, in the case where the detection result is fatigue driving, the fatigue level of the driving object can be determined based on the fatigue degree, and the early warning measures corresponding to the fatigue level are executed. For example, a voice prompt is given when the fatigue level is mild fatigue; seat vibration and steering wheel jitter are performed when the fatigue level is moderate fatigue; the hazard lights are turned on and the vehicle automatically decelerates when the fatigue level is severe fatigue. After decelerating, the vehicle automatically changes to the low-speed lane until it stops, or takes over the vehicle control right to ensure driving safety.

[0060] In the above technical solution, by executing the corresponding early warning measures based on the fatigue level, timely reminders can be made to ensure the driving safety of the driving object.

[0061] Figure 3 This is another structural block diagram provided by the embodiments of the present invention. This embodiment is optimized based on the above technical solutions. In this embodiment, optionally, the image processing module is further configured to send a first start signal for starting the fatigue detection module to the fatigue detection module based on the facial features; the sound processing module is further configured to send a second start signal for starting the fatigue detection module to the fatigue detection module based on the sound features; the fatigue detection module is specifically configured to, in the case of receiving at least one of the first start signal and the second start signal, obtain the facial features from the image processing module, and obtain the sound features from the sound processing module, and perform fatigue detection on the driving object based on the facial features and the sound features to obtain a detection result. Among them, the explanations of the same or corresponding terms as those in the above embodiments are not repeated here.

[0062] Specifically, refer to Figure 3, the fatigue driving detection system of this embodiment includes: an object information acquisition module 31, an image processing module 32, an audio processing module 33, and a fatigue detection module 34; where

[0063] The object information acquisition module 31 is configured to collect the image information and audio information of the driving object driving the target vehicle during the driving process of the target vehicle, send the image information to the image processing module 32, and send the audio information to the audio processing module 33;

[0064] The image processing module 32 is configured to extract features from the received image information to obtain the facial features of the driving object, and send a first start signal for starting the fatigue detection module 34 to the fatigue detection module 34 based on the facial features;

[0065] The audio processing module 33 is configured to extract features from the received audio information to obtain the voice features of the driving object, and send a second start signal for starting the fatigue detection module 34 to the fatigue detection module 34 based on the voice features;

[0066] The fatigue detection module 34 is configured to, when receiving at least one of the first start signal and the second start signal, obtain the facial features from the image processing module 32, and obtain the voice features from the audio processing module 33, and perform fatigue detection on the driving object based on the facial features and the voice features to obtain a detection result.

[0067] Among them, the first start signal can be understood as a signal for starting the fatigue detection module 34 to perform fatigue detection based on the facial features. Optionally, when it is determined through the facial features that the driving object may be fatigued while driving, the image processing module 32 can send the first start signal to enable the fatigue detection module 34 to start fatigue detection.

[0068] The second start signal can be understood as a signal for starting the fatigue detection module 34 to perform fatigue detection based on the voice features. Optionally, when it is determined through the voice features that the driving object may be fatigued while driving, the audio processing module 33 can send the second start signal to enable the fatigue detection module 34 to start fatigue detection.

[0069] The fatigue detection module 34 can perform fatigue detection on the driving object when receiving at least one of the first start signal and the second start signal, and obtain a detection result to avoid accidents due to untimely detection. Optionally, in order to further improve the timeliness of the fatigue detection module 34 to start fatigue detection, the object information acquisition module 31 can also collect the grip force exerted by the driving object on the steering wheel of the target vehicle, and send the grip force to the third start signal sending module. The third start signal sending module can send a third start signal to the fatigue detection module 34 when the received grip force value is less than a preset grip force threshold. The fatigue detection module 34 can obtain facial features from the image processing module and voice features from the voice processing module when receiving at least one of the first start signal, the second start signal, and the third start signal, and perform fatigue detection on the driving object based on the facial features and voice features to obtain a detection result.

[0070] In the above technical solution, by sending a start signal to the fatigue detection module, fatigue detection can be started in a timely manner when there is a risk of fatigue driving, improving the timeliness of fatigue detection.

[0071] An optional technical solution is that the image processing module includes a facial feature extraction sub-module, a fatigue state determination sub-module, and a first start signal sending sub-module; among them, the facial feature extraction sub-module is used to extract features from the image information received within a first time period at the end of the first time period for each preset first time period, to obtain the facial features of the driving object; the fatigue state determination sub-module is used to determine whether the state of the driving object within the first time period corresponding to the facial features is a fatigue state based on the facial features; the first start signal sending sub-module is used to send a first start signal for starting the fatigue detection module to the fatigue detection module when the driving object shows a fatigue state in a continuous first preset number of first time periods.

[0072] Among them, the driving time of the driving object can be divided into multiple preset first time periods. For each preset first time period, the facial feature extraction sub-module extracts features from the image information received within the first time period at the end of the first time period to obtain the facial features of the driving object. Then, the fatigue state determination sub-module determines whether the state of the driving object within this first time period is a fatigue state based on the facial features. In this way, the fatigue state of the driving object in each preset first time period can be obtained. Finally, when the driving object shows a fatigue state in a continuous first preset number of first time periods, it indicates that the driving object is indeed in a fatigue state, and then the first start signal sending sub-module sends a first start signal to the fatigue detection module. In this way, the fatigue situation of the driving object can be comprehensively judged through the image features of multiple first time periods, avoiding misjudgment and causing unnecessary detection.

[0073] The above technical solution can avoid false detection caused by accidental situations.

[0074] Another optional technical solution is a sound processing module, including a sound feature extraction submodule, a sound fatigue feature judgment submodule and a second start signal sending submodule; wherein the sound feature extraction submodule is used to extract features of the sound information received in the second time period at the end of each preset second time period to obtain the sound features of the driving object; the sound fatigue feature judgment submodule is used to determine whether the state of the driving object in the second time period corresponding to the sound feature is a fatigue state based on the sound feature; the second start signal sending submodule is used to send a second start signal for starting the fatigue detection module to the fatigue detection module when the driving object is in a fatigue state for a second preset number of consecutive second time periods.

[0075] Among them, the driving time of the driving object can also be divided into a plurality of preset second time periods, and the second time period can be the same as or different from the first time period. For each preset second time period, the sound feature extraction submodule extracts the sound information received in the second time period at the end of the second time period to obtain the sound feature of the driving object. Afterwards, the sound fatigue feature judgment submodule determines whether there is a fatigue sound feature that characterizes the fatigue sound of the driving object in the sound feature, so as to determine whether the state of the driving object in the first time period is a fatigue state. In this way, the fatigue state of the driving object in each preset second time period can be obtained. Finally, when the driving object is in a fatigue state in a second preset number of second time periods, it means that the driving object is indeed in a fatigue state, and then the second start signal sending submodule sends a second start signal to the fatigue detection module. In this way, the fatigue of the driving object can be comprehensively judged by the sound features of multiple second time periods. For example, when the sound features of snoring appear in multiple second time periods, it means that the driving object is in fatigue driving. Thereby, misjudgment can be avoided, resulting in unnecessary detection.

[0076] The above technical solution can avoid false detection caused by accidental situations.

[0077] In order to better understand the above-mentioned technical solutions as a whole, the following is an exemplary description of the above-mentioned technical solutions in combination with specific examples. In this specific example, the object information acquisition module includes an information acquisition module and a sound acquisition module. The flowchart of the fatigue driving detection system is as follows: Figure 4 As shown, the specific steps are as follows:

[0078] 1. During the driving process of the target vehicle, the information acquisition module collects the image information of the driving object of the target vehicle and sends the image information to the image processing and analysis module (i.e., the image processing module). For each preset first time period, at the end of the first time period, the image processing and analysis module extracts features from the image information received within the first time period to obtain the facial features of the driving object. Then, based on the facial features, it determines whether the state of the driving object within this first time period is a fatigued state. If the driving object shows a fatigued state in a continuous first preset number of first time periods, it sends a first activation signal to the fatigue detection module.

[0079] 2. During the driving process of the target vehicle, the sound acquisition module collects the sound information of the driving object of the target vehicle and sends the sound information to the sound processing and analysis module (i.e., the sound processing module). For each preset second time period, at the end of the second time period, the sound processing and analysis module extracts features from the sound information received within the second time period to obtain the sound features of the driving object. Then, based on the sound features, it determines whether the state of the driving object within this second time period is a fatigued state. If the driving object shows a fatigued state in a continuous second preset number of first time periods, it sends a second activation signal to the fatigue detection module.

[0080] 3. After receiving the first activation signal and / or the second activation signal, the fatigue detection module inputs the obtained facial features and sound features into a pre-trained fatigue degree determination model to obtain the fatigue degree of the driving object, and determines whether the fatigue degree is greater than a preset fatigue threshold. If not, no action is taken. If so, it sends the fatigue degree to the warning prompt module (i.e., the warning module). The warning prompt module determines the fatigue level of the driving object based on the fatigue degree and executes the warning measures corresponding to the fatigue level.

[0081] 4. The data mechanism and transmission module (i.e., the data recording module) obtains and uploads the image features, sound features, and detection results to the vehicle data center, so that the vehicle data center updates the preset fatigue threshold based on the image features, sound features, and detection results.

[0082] The above specific example can achieve accurate detection of the driving object's fatigue driving.

[0083] The above specific implementation manners do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A fatigue driving detection system, characterized in that: include: Object information acquisition module, image processing module, sound processing module and fatigue detection module; wherein, The object information acquisition module is used to collect image information and sound information of the driving object driving the target vehicle during the driving process of the target vehicle, and send the image information to the image processing module, and send the sound information to the sound processing module; The image processing module is used to extract features from the received image information to obtain facial features of the driving object; The sound processing module is used to extract features from the received sound information to obtain sound features of the driving object; The fatigue detection module is used to obtain the facial features from the image processing module and the sound features from the sound processing module, and perform fatigue detection on the driving object based on the facial features and the sound features to obtain a detection result.

2. The system according to claim 1, characterized in that The fatigue detection module includes a feature acquisition submodule and a fatigue detection submodule; wherein, The feature acquisition submodule is used to acquire the facial features from the image processing module and acquire the sound features from the sound processing module; The fatigue detection submodule is used to determine the fatigue degree of the driving object based on the acquired facial features and the voice features, and perform fatigue detection on the driving object based on the fatigue degree to obtain a detection result.

3. The system according to claim 2, characterized in that: The fatigue detection submodule includes a fatigue degree determination unit and a fatigue driving judgment unit; wherein, The fatigue level determination unit is used to input the acquired facial features and voice features into a pre-trained fatigue level determination model to obtain the fatigue level of the driving subject; The fatigue driving judgment unit is used to perform fatigue detection on the driving object and obtain a detection result according to whether the fatigue degree is greater than a preset fatigue threshold.

4. The system according to claim 3, characterized in that Also includes: A data recording module is used to obtain and upload the image features, the sound features and the detection results to a vehicle data center, so that the vehicle data center updates the preset fatigue threshold based on the image features, the sound features and the detection results, and sends the updated preset fatigue threshold to the fatigue driving judgment unit.

5. The system according to claim 2, characterized in that Also includes: The early warning module is used to determine the fatigue level of the driving object based on the fatigue degree when the detection result is fatigue driving, and to execute early warning measures corresponding to the fatigue level.

6. The system according to claim 1, characterized in that The image processing module is further used to send a first start signal for starting the fatigue detection module to the fatigue detection module based on the facial features; The sound processing module is further used to send a second start signal for starting the fatigue detection module to the fatigue detection module based on the sound feature; The fatigue detection module is specifically used to obtain the facial features from the image processing module and the sound features from the sound processing module when at least one of the first start signal and the second start signal is received, and to perform fatigue detection on the driving object based on the facial features and the sound features to obtain a detection result.

7. The system according to claim 6, characterized in that: The image processing module includes a facial feature extraction submodule, a fatigue state determination submodule and a first start signal sending submodule; wherein, The facial feature extraction submodule is used to extract features of the image information received in each preset first time period at the end of the first time period to obtain the facial features of the driving object; The fatigue state determination submodule is used to determine whether the state of the driving object in the first time period corresponding to the facial feature is a fatigue state based on the facial feature; The first start signal sending submodule is used to send a first start signal for starting the fatigue detection module to the fatigue detection module when the driving object exhibits the fatigue state within a first preset number of consecutive first time periods.

8. The system according to claim 6, characterized in that: The sound processing module includes a sound feature extraction submodule, a sound fatigue feature judgment submodule and a second start signal sending submodule; wherein, The sound feature extraction submodule is used for, for each preset second time period, at the end of the second time period, performing feature extraction on the sound information received in the second time period to obtain the sound feature of the driving object; The sound fatigue feature judgment submodule is used to determine whether the state of the driving object in the second time period corresponding to the sound feature is a fatigue state based on the sound feature; The second start signal sending submodule is used to send a second start signal for starting the fatigue detection module to the fatigue detection module when the driving object exhibits the fatigue state within a second preset number of consecutive second time periods.

9. The system according to claim 1, characterized in that The target vehicle is provided with a plurality of sound collection devices, and the object information collection module is further used for: Based on the time when the collected sound information arrives at each of the sound collecting devices and the installation positions of the multiple sound collecting devices, the sound source position of the sound source corresponding to the sound information is determined, so that when the sound source position is within the preset driving position range of the target vehicle, the sound information is sent to the sound processing module.

10. The system according to claim 1, characterized in that Also includes: The third start signal sending module; wherein, The object information acquisition module is further used to collect the grip force applied by the driving object on the steering wheel of the target vehicle, and send the grip force to the third start signal sending module; The third start signal sending module is used to send a third start signal to the fatigue detection module when the received grip strength value is less than a preset grip strength threshold; The fatigue detection module is also used to obtain the facial features from the image processing module and the sound features from the sound processing module when the third start signal is received, and perform fatigue detection on the driving object based on the facial features and the sound features to obtain a detection result.

Images