Fatigue driving detection method, device, readable medium and electronic device

Abstract

The present invention discloses a method, device, readable medium and electronic device for detecting fatigue driving, comprising: acquiring an image to be detected, and determining a pressure map to be detected corresponding to the image to be detected; using a predetermined fatigue detection model, determining a fatigue detection result based on the image to be detected and the pressure map to be detected; the present invention completes the detection of fatigue driving by combining the image to be detected and the pressure map to be detected, can detect the occurrence of fatigue driving in a timely manner, solves the hysteresis problem of the existing technology, and further improves driving safety.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, readable medium and electronic device for detecting fatigue driving. Background Art

[0002] Many new intelligent vehicles now have fatigue detection capabilities. This feature can analyze and determine the driver's focus and mental state while driving, thereby preventing fatigue and further improving driving safety.

[0003] Existing fatigue driving detection methods are mostly based on facial image analysis technology. This involves capturing a driver's facial image and using facial features such as expression and demeanor to determine whether the driver is focused and fatigued.

[0004] However, in reality, if a driver's facial features can be used to identify fatigue driving, then fatigue driving has likely already been going on for a considerable period of time. This means that current fatigue driving detection systems have a significant lag and cannot immediately detect fatigue driving. This safety issue requires further improvement. Summary of the Invention

[0005] The present invention provides a method, device, readable medium and electronic device for detecting fatigue driving, which realize real-time fatigue driving detection by combining an image to be detected with a corresponding pressure map to be detected.

[0006] In a first aspect, the present invention provides a method for detecting fatigue driving, comprising:

[0007] Acquire an image to be detected, and determine a pressure map to be detected corresponding to the image to be detected;

[0008] A fatigue detection result is determined by using a predetermined fatigue detection model according to the image to be detected and the pressure map to be detected.

[0009] Preferably, determining the pressure map to be detected corresponding to the image to be detected includes:

[0010] Determining a time range for acquiring the image to be detected;

[0011] determining a value of the pressure sensing data within the time range;

[0012] The pressure map to be detected is determined according to the value of the pressure sensing data.

[0013] Preferably, it also includes:

[0014] The fatigue detection model is predetermined by using contrastive learning data training.

[0015] Preferably, the utilizing contrastive learning data training to predetermine the fatigue detection model comprises:

[0016] determining a sample image and a sample pressure map;

[0017] Determine the initial model;

[0018] Using the sample image and the sample pressure map, performing the comparative learning data training on the initial model;

[0019] The trained initial model is determined as the fatigue detection model.

[0020] Preferably, the sample images include true sample images and false sample images; the sample pressure maps include true sample pressure maps and false sample pressure maps;

[0021] The performing the comparative learning data training on the initial model using the sample image and the sample pressure map includes:

[0022] Determining the true sample image and the true sample pressure map as a first positive sample pair;

[0023] Determining the fake sample image and the true sample pressure map as a second positive sample pair;

[0024] Determine the true sample image and the false sample pressure map as a first negative sample pair;

[0025] Determining the fake sample image and the fake sample pressure map as a second negative sample pair;

[0026] The contrastive learning data training is performed on the initial model using the first positive sample pair, the second positive sample pair, the first negative sample pair, and the second negative sample pair.

[0027] Preferably, the performing the comparative learning data training on the initial model includes:

[0028] Inputting the sample image and the sample pressure map into the initial model so that the initial model outputs a calculation result;

[0029] Determining the loss index of the calculation result using a preset loss function;

[0030] When the loss indicator meets a first preset condition, internal parameters of the initial model are adjusted.

[0031] Preferably, it also includes:

[0032] When the fatigue detection result meets the second preset condition, a warning message is pushed.

[0033] In a second aspect, the present invention provides a device for detecting fatigue driving, comprising:

[0034] A data acquisition module is used to acquire an image to be detected and determine a pressure map to be detected corresponding to the image to be detected;

[0035] The detection module is used to determine a fatigue detection result according to the image to be detected and the pressure map to be detected by using a predetermined fatigue detection model.

[0036] In a third aspect, the present invention provides a readable medium comprising an execution instruction. When a processor of an electronic device executes the execution instruction, the electronic device executes any method described in the first aspect.

[0037] In a fourth aspect, the present invention provides an electronic device comprising a processor and a memory storing execution instructions, wherein when the processor executes the execution instructions stored in the memory, the processor executes any method described in the first aspect.

[0038] The present invention provides a method, device, readable medium and electronic device for detecting fatigue driving, which combine the image to be detected and the pressure map to be detected to jointly complete the detection of fatigue driving. It can detect the occurrence of fatigue driving in a timely manner, solve the lag problem of the existing technology, and further improve driving safety.

[0039] The further effects of the above-mentioned non-conventional preferred embodiment will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the existing technical solutions, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 A flowchart of a method for detecting fatigue driving provided by one embodiment of the present invention;

[0042] Figure 2 A schematic diagram of a process for training a fatigue detection model in a fatigue driving detection method provided by one embodiment of the present invention;

[0043] Figure 3 A schematic structural diagram of a fatigue driving detection device provided by one embodiment of the present invention;

[0044] Figure 4 The present invention provides a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0046] Existing fatigue driving detection methods are mostly based on facial image analysis technology. Specifically, facial images of the driver are captured and analyzed to identify facial features such as expression and demeanor, thereby determining whether the driver is focused and fatigued. For example, detecting features such as squinting or yawning in the driver's eyes may indicate fatigue. Alternatively, detecting features such as talking or talking on the phone may indicate a lack of focus.

[0047] However, in reality, if a driver's facial features can be used to identify fatigue, then the driver's fatigue has likely persisted for a considerable period of time. However, existing technologies are unable to effectively detect fatigue during this period, nor can they effectively mitigate the risks. In other words, current fatigue detection methods suffer from a significant lag and fail to immediately detect fatigue. Further improvements are needed to ensure safety.

[0048] In view of this, the present invention provides a method for detecting fatigue driving. Figure 1 FIG. 1 is a specific embodiment of the method for detecting fatigue driving provided by the present invention. In this embodiment, the method includes:

[0049] Step 101: Acquire an image to be detected and determine a pressure map to be detected corresponding to the image to be detected.

[0050] The image to be detected is the face image to be detected in this embodiment. In this embodiment, it can be collected by a vehicle-mounted RGB camera. The pressure map to be detected can be determined based on the pressure sensing data collected by the pressure sensor mounted on the steering wheel of the vehicle. Specifically, the time range for collecting the image to be detected is determined. Then, the value of the pressure sensing data collected by the pressure sensor within this time range is obtained. In other words, it is to determine the pressure sensing data obtained by the vehicle-mounted pressure sensor and the change in the data during the process of collecting the image to be detected. The above-mentioned pressure sensing data can reflect the grip strength of the driver holding the steering wheel when collecting the image to be detected. Then, based on the value of the pressure sensing data within the above-mentioned time range, the pressure map to be detected can be determined.

[0051] Typically, the pressure map to be detected is time series data, that is, it reflects the change of the pressure sensor data value over time within the time range. From a visualization perspective, the pressure map to be detected can be a two-dimensional curve, with the horizontal axis representing time and the vertical axis representing the data value.

[0052] It's understandable that when a driver is in a normal or fatigued state, not only will their facial features differ, but their grip on the steering wheel will also vary. Typically, during fatigued driving or when their attention is distracted, the driver's grip on the steering wheel will noticeably decrease. The pressure sensor data collected will reflect this change in grip force. Therefore, theoretically, changes in grip force during fatigued driving will be reflected in the pressure map to be detected.

[0053] Step 102: Using a predetermined fatigue detection model, a fatigue detection result is determined according to the image to be detected and the pressure map to be detected.

[0054] In this embodiment, the fatigue detection model is a computational model built based on artificial intelligence technology. Specifically, the fatigue detection model can be pre-determined using comparative learning data training. This embodiment does not limit the specific training process for the fatigue detection model. Any training process based on the same or similar principles may be incorporated into the overall technical solution of this embodiment.

[0055] As previously known, changes in the driver's grip force during fatigued driving are reflected in the pressure map to be detected. This means that there are certain underlying patterns between the image to be detected and the pressure map to be detected, depending on whether the driver is in a normal or fatigued state. A fatigue detection model built based on artificial intelligence technology can grasp this underlying pattern to a certain extent during the data training process. In subsequent applications, it can then be used to calculate and determine whether the input image to be detected and the pressure map to be detected represent the driver's normal state or fatigue. This allows the driver to be detected whether they are driving fatigued and whether they are distracted or inattentive.

[0056] Therefore, the input data for the fatigue detection model is the image to be detected and the pressure map to be detected. Through the internal operations of the fatigue detection model, the corresponding fatigue detection result can be calculated based on the input image to be detected and the pressure map to be detected. The fatigue detection result can specifically be "normal driving" or "fatigue driving".

[0057] In addition, if the fatigue detection result is "fatigue driving", it can be considered that the fatigue detection result meets the second preset condition. At this time, a warning message can be further pushed to remind the driver to concentrate or take a break in time to avoid the danger of fatigue driving.

[0058] This embodiment combines the image to be detected and the pressure map to be detected to detect fatigue driving. As can be seen, the detection results, like those of existing technologies, reflect the analysis of facial features. Furthermore, they utilize the changing grip strength of drivers during fatigue driving. This change in grip strength is an instinctive human reaction to fatigue, and is immediately apparent and accounted for by the detection method of this embodiment.

[0059] It can be seen from the above technical solution that the beneficial effect of this embodiment is: the detection of fatigue driving is completed by combining the image to be detected and the pressure map to be detected, and the occurrence of fatigue driving can be detected in time, which solves the lag problem of the existing technology and further improves driving safety.

[0060] Figure 1 What is shown is only a basic embodiment of the method of the present invention. By performing certain optimization and expansion on this basis, other preferred embodiments of the method can be obtained.

[0061] like Figure 2 FIG. 1 is a flow chart of a fatigue detection model training process in a face detection method according to the present invention. In this embodiment, the training process of the fatigue detection model will be specifically described, which specifically includes the following steps:

[0062] Step 201: Determine a sample image and a sample pressure map.

[0063] In this embodiment, a fatigue detection model is obtained by training using contrastive learning training. A certain number of sample images and sample pressure maps are required during the training process. The sample image is a facial image captured using an RGB camera. The sample pressure map also corresponds to the time range in which the sample image is captured. However, it should be noted that in order to meet the requirements of the corresponding samples for contrastive learning training, the sample images include true sample images and false sample images. Among them, the true sample image is a facial image from a fatigued state, and the false sample image is a facial image from a non-fatigued state. The sample pressure map also includes true sample pressure maps and false sample pressure maps. The true sample pressure map is real and corresponds to the acquisition time of the sample image, and the false sample pressure map does not correspond to the acquisition time of the sample image.

[0064] Thus, the true sample image and the true sample pressure map can be determined as the first positive sample pair. The false sample image and the true sample pressure map can be determined as the second positive sample pair. The true sample image and the false sample pressure map can be determined as the first negative sample pair. The false sample image and the false sample pressure map can be determined as the second negative sample pair.

[0065] In other words, the positive sample pairs (including the first positive sample pair and the second positive sample pair) demonstrate a true correspondence between the sample image and the sample pressure map (regardless of whether the sample image is forged). In contrast, the negative sample pairs (including the first negative sample pair and the second negative sample pair) demonstrate an untrue correspondence between the sample image and the sample pressure map. This comparative learning allows the trained fatigue detection model to uncover regularities between the sample images and the sample pressure map, thereby further enabling fatigue state detection.

[0066] Step 202: Determine the initial model.

[0067] The initial model is an untrained network model whose structural characteristics can be applied to contrastive learning training. The specific structure of the initial model is not limited in this embodiment. Any network model with the above characteristics in the art can be combined with the overall solution of this embodiment.

[0068] Step 203: Use the sample images and sample pressure maps to perform comparative learning data training on the initial model.

[0069] In this embodiment, the initial model is trained using the aforementioned first positive sample pair, second positive sample pair, first negative sample pair, and second negative sample pair for comparative learning data. The training process can be described as follows: a sample image and a sample pressure map are input into the initial model, causing the initial model to output a computation result. This computation result can determine whether the input data is a positive or negative sample pair, or whether the sample image is true or false. A pre-set loss function is then used to determine a loss index for the computation result. The specific content of the loss function is not limited in this embodiment; any loss function known in the art that can perform the same or similar function can be incorporated into the overall solution of this embodiment. The so-called loss index, to a certain extent, reflects the accuracy of the computation result. When the loss index meets a first preset condition, the internal parameters of the initial model are adjusted. If the loss index meets the first preset condition, this indicates that the accuracy of the computation result is still low, indicating that the computation result of the initial model is inaccurate. At this point, the internal parameters of the initial model are adjusted, and the training process is repeated until the loss index no longer meets the first preset condition, at which point the training is considered complete.

[0070] Step 204: Determine the trained initial model as the fatigue detection model.

[0071] The initial model after training, i.e. Figure 1 The fatigue detection model used in the illustrated embodiment can be run in the form of a software program and implemented on an onboard intelligent system to realize a fatigue detection function.

[0072] like Figure 3 The figure shows a specific embodiment of the fatigue driving detection device of the present invention. The device described in this embodiment is used to perform Figures 1-2 The physical device of the method. Its technical solution is essentially consistent with the above embodiment, and the corresponding description in the above embodiment is also applicable to this embodiment. The device in this embodiment includes:

[0073] The data acquisition module 301 is used to acquire an image to be detected and determine a pressure map to be detected corresponding to the image to be detected.

[0074] The detection module 302 is configured to use a predetermined fatigue detection model to determine a fatigue detection result according to the image to be detected and the pressure map to be detected.

[0075] In addition Figure 3 Based on the embodiment shown, preferably, the present invention further includes:

[0076] The data acquisition module 301 includes:

[0077] The image acquisition unit 311 is used to acquire the image to be detected.

[0078] The pressure map acquisition unit 312 is used to determine a time range for acquiring an image to be detected; determine a value of the pressure sensing data within the time range; and determine a pressure map to be detected based on the value of the pressure sensing data.

[0079] Also includes:

[0080] The training module 303 is used to train using contrastive learning data to predetermine a fatigue detection model.

[0081] The training module 303 includes:

[0082] The sample determination unit 331 is configured to determine a sample image and a sample pressure map.

[0083] The initial model determining unit 332 is configured to determine an initial model.

[0084] The model training unit 333 is used to perform comparative learning data training on the initial model using the sample images and the sample pressure maps, and determine the trained initial model as the fatigue detection model.

[0085] The sample images include true sample images and false sample images; the sample pressure maps include true sample pressure maps and false sample pressure maps; the sample determination unit 341 includes:

[0086] The first positive sample pair determining subunit 3411 is configured to determine the true sample image and the true sample pressure map as a first positive sample pair.

[0087] The second positive sample pair determination subunit 3412 is configured to determine the false sample image and the true sample pressure map as a second positive sample pair.

[0088] The first negative sample pair determining subunit 3413 is configured to determine the true sample image and the false sample pressure map as a first negative sample pair.

[0089] The second negative sample pair determining subunit 3414 is configured to determine the fake sample image and the fake sample pressure map as a second negative sample pair.

[0090] The model training unit 333 includes:

[0091] The operation subunit 3331 is used to input the sample image and the sample pressure map into the initial model so that the initial model outputs the operation result.

[0092] The loss calculation subunit 3332 is used to determine the loss index of the calculation result using a preset loss function.

[0093] The parameter adjustment subunit 3333 is used to adjust the internal parameters of the initial model when the loss index meets the second preset condition.

[0094] Also includes:

[0095] The warning module 304 is configured to push a warning message when the fatigue detection result meets a second preset condition.

[0096] Figure 4 : This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include internal memory, such as high-speed random access memory (RAM), and may also include non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0097] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0098] Memory is used to store execution instructions. Specifically, execution instructions are computer programs that can be executed. Memory can include internal memory and non-volatile memory, and provides execution instructions and data to the processor.

[0099] In one possible implementation, a processor reads corresponding execution instructions from non-volatile memory into internal memory and then executes them. Alternatively, the processor can obtain corresponding execution instructions from other devices to form a fatigue driving detection device at a logical level. The processor executes the execution instructions stored in the memory to implement the fatigue driving detection method provided in any embodiment of the present invention.

[0100] The present invention Figure 3The method performed by the fatigue driving detection device provided in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0101] The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules within the decoding processor. The software modules can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the method described above.

[0102] The embodiment of the present invention further provides a readable medium, which stores an execution instruction. When the stored execution instruction is executed by the processor of the electronic device, the electronic device can execute the fatigue driving detection method provided in any embodiment of the present invention, and is specifically used to execute the following Figure 1 or Figure 2 The method shown.

[0103] The electronic device described in each of the aforementioned embodiments may be a computer.

[0104] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or a combination of software and hardware.

[0105] The various embodiments of the present invention are described in a progressive manner. Similar portions between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so their description is relatively simple. For relevant portions, refer to the description of the method embodiments.

[0106] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0107] The foregoing is merely an embodiment of the present invention and is not intended to limit the present invention. It will be apparent to those skilled in the art that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A method for detecting fatigue driving, characterized in that: include: Using contrastive learning data training to predetermine the fatigue detection model; Acquire an image to be detected, and determine a pressure map to be detected corresponding to the image to be detected; The image to be detected is a face image to be detected; the pressure map to be detected is determined based on pressure sensing data collected by a pressure sensor mounted on a steering wheel of the vehicle; Determining a fatigue detection result based on the image to be detected and the pressure map to be detected using a predetermined fatigue detection model; Using contrastive learning data training to predetermine a fatigue detection model includes: determining a sample image and a sample pressure map; determining an initial model; using the sample image and the sample pressure map to train the initial model with the contrastive learning data; and determining the trained initial model as the fatigue detection model; The sample images include true sample images and false sample images; the sample pressure maps include true sample pressure maps and false sample pressure maps; the use of the sample images and the sample pressure maps to perform the comparative learning data training on the initial model includes: determining the true sample image and the true sample pressure map as a first positive sample pair; determining the false sample image and the true sample pressure map as a second positive sample pair; determining the true sample image and the false sample pressure map as a first negative sample pair; determining the false sample image and the false sample pressure map as a second negative sample pair; and using the first positive sample pair, the second positive sample pair, the first negative sample pair and the second negative sample pair to perform the comparative learning data training on the initial model.

2. The method according to claim 1, characterized in that Determining the pressure map to be detected corresponding to the image to be detected includes: Determining a time range for acquiring the image to be detected; determining a value of the pressure sensing data within the time range; The pressure map to be detected is determined according to the value of the pressure sensing data.

3. The method according to claim 1, characterized in that The performing the comparative learning data training on the initial model includes: Inputting the sample image and the sample pressure map into the initial model so that the initial model outputs a calculation result; Determining the loss index of the calculation result using a preset loss function; When the loss indicator meets a first preset condition, internal parameters of the initial model are adjusted.

4. The method according to any one of claims 1 to 3, characterized in that Also includes: When the fatigue detection result meets the second preset condition, a warning message is pushed.

5. A fatigue driving detection device, characterized in that: include: A training module for training and predetermining a fatigue detection model using contrastive learning data; A data acquisition module is configured to acquire an image to be detected and determine a pressure map to be detected corresponding to the image to be detected; the image to be detected is a face image to be detected; the pressure map to be detected is determined based on pressure sensing data acquired by a pressure sensor mounted on a steering wheel of the vehicle; a detection module, configured to determine a fatigue detection result based on the image to be detected and the pressure map to be detected by using a predetermined fatigue detection model; The training module includes: a sample determination unit for determining a sample image and a sample pressure map; an initial model determination unit for determining an initial model; a model training unit for performing comparative learning data training on the initial model using the sample image and the sample pressure map, and determining the trained initial model as a fatigue detection model; The sample image includes a true sample image and a false sample image; the sample pressure map includes a true sample pressure map and a false sample pressure map; the sample determination unit includes: a first positive sample pair determination subunit, used to determine the true sample image and the true sample pressure map as a first positive sample pair; a second positive sample pair determination subunit, used to determine the false sample image and the true sample pressure map as a second positive sample pair; a first negative sample pair determination subunit, used to determine the true sample image and the false sample pressure map as a first negative sample pair; and a second negative sample pair determination subunit, used to determine the false sample image and the false sample pressure map as a second negative sample pair. The model training unit is specifically configured to perform the comparative learning data training on the initial model using the first positive sample pair, the second positive sample pair, the first negative sample pair, and the second negative sample pair.

6. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the fatigue driving detection method according to any one of claims 1 to 4.

7. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the fatigue driving detection method described in any one of claims 1 to 4 above.

Images