Fatigue driving detection method, device, electronic equipment and medium

Abstract

The present application provides a method, device, electronic device and medium for detecting fatigue driving, wherein the method comprises: inputting the acquired driving monitoring image of the service provider into a facial feature point detection model to determine the facial feature points; intercepting a feature area picture from the driving monitoring image according to the position of the facial feature points; inputting the feature area picture into a fatigue detection model to obtain the fatigue level of the service provider. The facial feature points determined by the facial feature point detection model used in the present application do not need to be particularly accurate, and the facial feature points are only used as a reference position to intercept the feature area picture. The intercepted feature picture is input into the fatigue detection model to obtain the fatigue level of the service provider, thereby improving the accuracy of determining the fatigue level of the service provider when the acquired driving monitoring image is not a standard image.

Description

Technical Field

[0001] The present application relates to the field of signal processing technology, and in particular to a method, device, electronic device and medium for detecting fatigue driving. Background Art

[0002] At present, with the rapid development of the Internet, people's pace of life is getting faster and faster. More and more people choose to drive to improve travel efficiency. However, drivers will become fatigued after driving for a long time. In order to improve the travel safety of drivers and passengers, it is necessary to monitor the driver's fatigue and give reminders to the driver in time.

[0003] Generally, to monitor the driver's fatigue, a camera needs to be set up in front of the driver to obtain the driver's image in real time, and a detection model is used to determine the points of the left upper eyelid, the left lower eyelid, the right upper eyelid, the right lower eyelid, the upper lip and the lower lip in the driving monitoring image, and then the driver's fatigue is determined based on the distance between the points of the upper eyelids and the lower eyelids of the two eyes, and the distance between the points of the upper lip and the lower lip. However, in this way of determining the driver's fatigue, the camera needs to obtain the front driving monitoring image of the driver. In the case of deviation in the driver's driving monitoring image, the driver's fatigue detection result is inaccurate. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a method, device, electronic device and medium for detecting fatigue driving, which improves the accuracy of determining the fatigue level of the service provider when the acquired driving monitoring image is not a standard image.

[0005] According to one aspect of the present application, an electronic device is provided, which may include a storage medium and a processor communicating with the storage medium. The storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via a bus, and the processor executes the machine-readable instructions to perform the following operations:

[0006] In a first aspect, an embodiment of the present application provides a method for detecting fatigue driving, comprising:

[0007] Inputting the acquired driving monitoring image of the service provider into a facial feature point detection model to determine facial feature points;

[0008] According to the positions of the facial feature points, intercepting a feature area picture from the driving monitoring image;

[0009] The feature area image is input into a fatigue detection model to obtain the fatigue level of the service provider.

[0010] In some embodiments, the acquired driving monitoring image of the service provider is input into a facial feature point detection model to determine facial feature points, including:

[0011] For each moment, inputting the driving monitoring image of the service provider at that moment into the facial feature determination model to obtain the facial feature points corresponding to that moment;

[0012] According to the location of facial feature points, the feature area image is captured from the driving monitoring image, including:

[0013] For each moment, according to the corresponding facial feature points, a feature area image of the moment is cut out from the driving monitoring image of the moment.

[0014] In some embodiments, the feature area image is input into a fatigue detection model to obtain the fatigue level of the service provider, including:

[0015] For each moment, the captured feature area image is input into the fatigue detection model to obtain the reference fatigue value at that moment;

[0016] The fatigue degree of the service provider is determined according to the reference fatigue value at each moment.

[0017] In some embodiments, the facial landmarks include one or more of the following locations:

[0018] Left eye feature points, right eye feature points and mouth feature points;

[0019] The left eye feature points include: the position of the center point of the left eye, the position of the tail of the left eye and the position of the corner of the left eye;

[0020] The right eye feature points include: the position of the center point of the right eye, the position of the end of the right eye and the position of the corner of the right eye;

[0021] The mouth feature points include: the position of the mouth center point, the position of the left mouth corner and the position of the right mouth corner.

[0022] In some embodiments, the feature area image includes any one or more of the following images:

[0023] Left eye area image, right eye area image and mouth area image.

[0024] In some embodiments, when a plurality of characteristic region images are captured, the characteristic region images are input into a fatigue detection model to obtain the fatigue degree of the service provider, including:

[0025] For each feature region image, input the feature region image into a corresponding fatigue detection model to determine a fatigue reference value corresponding to the feature region image;

[0026] If the fatigue reference value corresponding to at least one regional characteristic image exceeds a preset value, it is determined that the fatigue level of the service provider is in a fatigue state.

[0027] In some embodiments, it also includes:

[0028] A processing strategy is determined according to the fatigue level of the service provider.

[0029] In some embodiments, the processing strategy includes one or more of the following processing methods:

[0030] Generate a prompt instruction for the service provider and stop dispatching service orders to the service provider.

[0031] In some embodiments, the prompt instruction includes any one or more of the following instructions:

[0032] Voice prompt commands, steering wheel vibration commands and seat vibration commands.

[0033] In some embodiments, determining a processing strategy according to the fatigue level of the service provider includes:

[0034] When the fatigue level is greater than a first preset value, and the fatigue level is less than or equal to a second preset value, executing a first processing strategy; wherein the first processing strategy uses a non-compulsory processing method to allow the service provider to rest;

[0035] When the fatigue level is greater than a second preset value, a second processing strategy is executed; wherein the first processing strategy uses a forced processing method to make the service provider take a rest.

[0036] In some embodiments, the first processing strategy includes generating a prompt instruction for the service provider;

[0037] The second processing strategy includes stopping dispatching service orders to the service provider.

[0038] In some embodiments, when the fatigue detection model includes multiple models, inputting the feature area image into the fatigue detection model to obtain the fatigue level of the service provider includes:

[0039] Inputting the characteristic region image into a strong light fatigue detection model and a dark light fatigue detection model respectively to obtain output results of each fatigue detection model;

[0040] A weighted sum is performed according to the output result of each fatigue detection model and the weight value corresponding to each output result to obtain the fatigue degree of the service provider.

[0041] In some embodiments, the weight value corresponding to each output result is determined based on the current moment; or, the weight value corresponding to each output result is determined based on the amount of training sample data of the fatigue detection model corresponding to the output result.

[0042] In a second aspect, an embodiment of the present application provides a device for detecting fatigue driving, comprising:

[0043] A determination module, used for inputting the acquired driving monitoring image of the service provider into a facial feature point detection model to determine facial feature points;

[0044] A capture module, used for capturing a feature area image from the driving monitoring image according to the positions of the facial feature points;

[0045] The fatigue level determination module is used to input the feature area image into a fatigue detection model to obtain the fatigue level of the service provider.

[0046] In some embodiments, the determining module includes:

[0047] A first determination subunit, for each moment, is used to input the driving monitoring image of the service provider at that moment into the facial feature determination model to obtain facial feature points corresponding to that moment;

[0048] The interception module comprises:

[0049] The first interception subunit is used to intercept, for each moment, a feature area image at that moment from the driving monitoring image at that moment according to the corresponding facial feature points.

[0050] In some embodiments, the fatigue level determination module includes:

[0051] A first reference fatigue value determination unit, for each moment, is used to input the intercepted feature area image into the fatigue detection model to obtain a reference fatigue value at that moment;

[0052] The first fatigue level determination subunit is used to determine the fatigue level of the service provider according to the reference fatigue value at each moment.

[0053] In some embodiments, the facial landmarks include one or more of the following locations:

[0054] Left eye feature points, right eye feature points and mouth feature points;

[0055] The left eye feature points include: the position of the center point of the left eye, the position of the tail of the left eye and the position of the corner of the left eye;

[0056] The right eye feature points include: the position of the center point of the right eye, the position of the end of the right eye and the position of the corner of the right eye;

[0057] The mouth feature points include: the position of the mouth center point, the position of the left mouth corner and the position of the right mouth corner.

[0058] In some embodiments, the feature area image includes any one or more of the following images:

[0059] Left eye area image, right eye area image and mouth area image.

[0060] In some embodiments, when multiple characteristic region images are captured, the fatigue level determination module includes:

[0061] A second reference fatigue value determination unit, for each feature region image, is used to input the feature region image into a corresponding fatigue detection model to determine a fatigue reference value corresponding to the feature region image;

[0062] The second fatigue level determination subunit is configured to determine that the fatigue level of the service provider is a fatigue state if the fatigue reference value corresponding to at least one regional characteristic image exceeds a preset value.

[0063] In some embodiments, it also includes:

[0064] The processing module is used to determine a processing strategy according to the fatigue level of the service provider.

[0065] In some embodiments, the processing strategy includes one or more of the following processing methods:

[0066] Generate a prompt instruction for the service provider and stop dispatching service orders to the service provider.

[0067] In some embodiments, the prompt instruction includes any one or more of the following instructions:

[0068] Voice prompt commands, steering wheel vibration commands and seat vibration commands.

[0069] In some embodiments, the processing module comprises:

[0070] A first processing subunit is configured to execute a first processing strategy when the fatigue level is greater than a first preset value and the fatigue level is less than or equal to a second preset value; wherein the first processing strategy uses a non-compulsory processing method to allow the service provider to rest;

[0071] The second processing subunit is used to execute a second processing strategy when the fatigue level is greater than a second preset value; wherein the first processing strategy uses a forced processing method to make the service provider take a rest.

[0072] In some embodiments, the first processing strategy includes generating a prompt instruction for the service provider;

[0073] The second processing strategy includes stopping dispatching service orders to the service provider.

[0074] In some embodiments, when the fatigue detection model includes multiple models, the fatigue degree determination module includes:

[0075] An output result unit, used for inputting the characteristic region image into the strong light fatigue detection model and the dark light fatigue detection model respectively, so as to obtain the output result of each fatigue detection model;

[0076] The third fatigue level determination subunit is used to perform weighted summation according to the output results of each fatigue detection model and the weight value corresponding to each output result, so as to obtain the fatigue level of the service provider.

[0077] In some embodiments, the weight value corresponding to each output result is determined based on the current moment; or, the weight value corresponding to each output result is determined based on the amount of training sample data of the fatigue detection model corresponding to the output result.

[0078] In a third aspect, an embodiment of the present application provides a vehicle-mounted device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned fatigue driving detection method are implemented.

[0079] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the above-mentioned fatigue driving detection method.

[0080] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned fatigue driving detection method are executed.

[0081] The fatigue driving detection method, device, electronic device and medium provided in the embodiments of the present application first input the acquired driving monitoring image of the service provider into a facial feature point detection model to determine the facial feature points; then, based on the positions of the facial feature points, a feature area picture is captured from the driving monitoring image; finally, the feature area picture is input into the fatigue detection model to obtain the fatigue level of the service provider.

[0082] In a certain embodiment, in the method provided by the present application, the facial feature points determined by the facial feature point detection model do not need to be particularly accurate. The facial feature points are only used as a reference position to capture a feature area image. The captured feature image is input into the fatigue detection model to obtain the fatigue level of the service provider. Therefore, the accuracy of determining the fatigue level of the service provider is improved when the acquired driving monitoring image is not a standard image. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0084] Figure 1 A basic flow chart of a method for detecting fatigue driving provided in an embodiment of the present application is shown;

[0085] Figure 2 A basic flow chart for determining the fatigue level of a service provider provided by an embodiment of the present application is shown;

[0086] Figure 3 A block diagram of a service system in which a fatigue driving detection method according to some embodiments of the present application is located is shown;

[0087] Figure 4 A schematic diagram of the structure of a fatigue driving detection device provided in an embodiment of the present application is shown;

[0088] Figure 5 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0089] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0090] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0091] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0092] In order to improve travel efficiency, users generally choose online car-hailing, private cars, shared cars and other means of transportation. However, when using the above means of transportation, the driver's driving time is too long and the driver will become fatigued. The driver's fatigue driving may reduce the personal safety of the driver and passengers. In order to improve the travel safety of drivers and passengers, it is necessary to monitor the driver's fatigue and send reminders to the driver in time.

[0093] Taking online ride-hailing as an example, monitoring the service provider (driver) and determining the service provider's fatigue level includes the following steps:

[0094] Step 1, obtaining a driving monitoring image from a service provider;

[0095] Step 2, inputting the acquired driving monitoring image into the detection model to determine the points of the left upper eyelid, the left lower eyelid, the right upper eyelid, the right lower eyelid, the upper lip and the lower lip;

[0096] Step 3, determining the fatigue level of the service provider based on the distance between the point of the upper eyelid and the point of the lower eyelid of both eyes, and the distance between the point of the upper lip and the point of the lower lip.

[0097] Based on the determined fatigue level of the service provider, it is determined whether to issue a reminder to the service provider, so that the service provider can increase the awake time, thereby improving the personal safety of the service provider and the passengers.

[0098] In the process of determining the fatigue degree of the service provider, when obtaining the driving monitoring image of the service provider, the camera needs to maintain a relative relationship with the face of the service provider so that the driving monitoring image obtained by the camera is standard (the standard driving monitoring image is usually the front image of the service provider). Usually, facial feature points can only be accurately found in the standard driving monitoring image. The facial feature points can include one or more of the following positions: left eye feature points, right eye feature points and mouth feature points; left eye feature points include: the position of the center point of the left eye, the position of the end of the left eye and the position of the corner of the left eye; right eye feature points include: the position of the center point of the right eye, the position of the end of the right eye and the position of the corner of the right eye; mouth feature points include: the position of the center point of the mouth, the position of the left corner of the mouth and the position of the right corner of the mouth. However, when obtaining the driving monitoring image as described above, the side driving monitoring image of the service provider may be obtained due to the driving action of the service provider, the side driving monitoring image of the service provider may be obtained due to the installation position of the camera, etc. Since the obtained driving monitoring image is not standard, the fatigue degree of the service provider determined based on the non-standard driving monitoring image is inaccurate.

[0099] Therefore, the inventor of the present application believes that a facial feature point detection model can be used to obtain facial feature points with less precise positions, and feature area images can be obtained through the facial feature points. A fatigue detection model can be used to determine the fatigue level of the service provider corresponding to the feature area image. Therefore, some non-standard driving monitoring images can also accurately determine the fatigue level of the service provider, thereby improving the accuracy of determining the fatigue level of the service provider.

[0100] Based on this, Figure 1 As shown, the present application provides a method for detecting fatigue driving, comprising:

[0101] S101, inputting the acquired driving monitoring image of the service provider into a facial feature point detection model to determine facial feature points;

[0102] S102, capturing a feature area image from the driving monitoring image according to the positions of the facial feature points;

[0103] S103: Input the feature area image into a fatigue detection model to obtain the fatigue level of the service provider.

[0104] In step S101, the service provider may be a driver, such as a person driving an online car-hailing vehicle. The driving monitoring image may be an image of the driving area, the driving monitoring image may be a facial image containing the service provider, or may not contain the facial image of the service provider, and the driving monitoring image may be obtained by a shooting device, wherein the shooting device may be a vehicle-mounted camera (orange recorder), a mobile phone camera, etc., and the setting position of the shooting device may be a position where the face of the service provider in the driving area can be captured. The facial feature point detection model can detect facial feature points in the driving monitoring image by the user, and the facial feature points exist in the facial image in the driving monitoring image. The facial feature points may include one or more of the following positions: left eye feature points, right eye feature points, and mouth feature points; the left eye feature points include: the position of the center point of the left eye, the position of the tail of the left eye, and the position of the corner of the left eye; the right eye feature points include: the position of the center point of the right eye, the position of the tail of the right eye, and the position of the corner of the right eye; the mouth feature points include: the position of the center point of the mouth, the position of the left corner of the mouth, and the position of the right corner of the mouth.

[0105] Specifically, the server obtains the driving monitoring image of the service provider. In order to determine the facial feature points in the driving monitoring image, the obtained driving monitoring image needs to be input into the facial feature point detection model. Only after the facial feature points are determined can the subsequent steps S102 and S103 be executed.

[0106] In step S102, the feature area picture may be a picture that can be used as a feature to judge the fatigue level of the service provider. The features that can be used as features to judge the fatigue level of the service provider may include any one or more of the following: left eye, right eye and mouth. Therefore, the feature area picture may include any one or more of the following pictures: left eye area picture, right eye area picture and mouth area picture.

[0107] Specifically, a square can be made with the distance between the end of the left eye and the corner of the left eye as the side length and the center point of the left eye as the center, and the left eye area image can be intercepted with the square; or, a square can be made with the distance between the end of the right eye and the corner of the right eye as the side length and the center point of the right eye as the center, and the right eye area image can be intercepted with the square; or, a square can be made with the distance between the left corner of the mouth and the right corner of the mouth as the side length and the center point of the mouth as the center, and the mouth area image can be intercepted with the square. The facial feature points are only used as a reference position to intercept the feature area image, and there is no need to obtain a more precise value, which reduces the situation where the inaccurate determination of the fatigue level of the service provider due to inaccurate values ​​obtained.

[0108] In step S103, the fatigue detection model may be used to detect the fatigue degree of the service provider corresponding to the feature area image. The fatigue degree may be a standard used to characterize the fatigue condition of the service provider, and the fatigue degree may be a numerical value, a level, etc. Among them, the level of fatigue degree may be a normal state (not fatigued), mild fatigue, severe fatigue, etc., which may be determined according to actual conditions. In the fatigue detection model, the fatigue degree is determined according to the degree of opening and closing of the eyes or the degree of opening and closing of the mouth. The wider the eyes are opened, the less fatigue the service provider is, and the smaller the eyes are opened, the greater the fatigue degree of the service provider is. The wider the mouth is opened, the greater the fatigue degree of the service provider is, and the smaller the mouth is opened, the less fatigue the service provider is.

[0109] Specifically, after inputting the feature area image into the fatigue detection model, the fatigue detection model will obtain the value corresponding to the feature area image, and this value can be used as the fatigue level of the service provider, or the level corresponding to the value can be determined, and the determined fatigue level is the fatigue level of the service provider. By detecting the feature area image through the fatigue detection model, the fatigue level of the service provider is determined. It is not necessary to obtain particularly precise values ​​(for example, the distance of eye opening) to accurately determine the fatigue level of the service provider, thereby reducing the inability to accurately determine the fatigue level of the service provider when the driving monitoring image is not a standard image and the precise value cannot be obtained.

[0110] In the embodiment of the present application, three steps are adopted. First, the positions of multiple facial feature points in the driving monitoring image are determined using a facial feature point detection model. Then, a feature area picture is intercepted in the driving monitoring image according to the positions of the determined facial feature points. Finally, the intercepted feature area picture is input into a fatigue driving detection model to determine the fatigue degree of the service provider. The facial feature points determined by the facial feature point detection model used in the present application do not need to be particularly accurate. The facial feature points are only used as a reference position to intercept the feature area picture. The intercepted feature picture is input into the fatigue detection model to obtain the fatigue degree of the service provider. Therefore, the accuracy of determining the fatigue degree of the service provider is improved when the acquired driving monitoring image is not a standard image.

[0111] The fatigue level of the service provider determined by a driving monitoring image is only the fatigue level of the service provider at a certain moment. If the service provider is captured with his eyes closed at this moment, and the characteristic area image is a captured image of the service provider's eye area, then it may be determined that the service provider is in a high degree of fatigue. Therefore, it is a bit too one-sided to determine the fatigue level of the service provider by only one driving monitoring image. In order to improve the accuracy of determining the fatigue level of the service provider, multiple and continuous driving monitoring images can be used to determine the fatigue level of the service provider, that is, step S101, including:

[0112] Step 1011, for each moment, inputting the driving monitoring image of the service provider at that moment into the facial feature determination model to obtain the facial feature points corresponding to that moment;

[0113] Step S102 includes:

[0114] Step 1021, for each moment, based on the corresponding facial feature points, extracting a feature area image at that moment from the driving monitoring image at that moment;

[0115] Step S103 includes:

[0116] Step 1031, for each moment, inputting the captured feature area image into a fatigue detection model to obtain a reference fatigue value at that moment;

[0117] Step 1032: Determine the fatigue level of the service provider according to the reference fatigue value at each moment.

[0118] In step 1011, each moment is a moment when a driving monitoring image is acquired each time, and the time interval between two adjacent moments is preset, and the time interval between two moments can be 1 second, 2 seconds, 3 seconds, etc.

[0119] Specifically, the driving monitoring images acquired at each moment need to be input into the facial feature determination model to obtain the facial feature points corresponding to each moment. Only by obtaining the facial feature points corresponding to each moment can the subsequent steps 1021, 1031, and 1032 be executed.

[0120] In step 1021, the fatigue level of the service provider needs to be determined at each moment, so the driving monitoring image corresponding to each moment needs to capture the feature area image according to the facial feature points. In order to improve the safety of the service provider and the passengers riding the service provider, the fatigue level of the service provider needs to be monitored at all times. By monitoring the service provider at all times, the attention of the service provider can be improved, thereby improving the safety of the service provider and the passengers riding the service provider.

[0121] In step 1031 , the fatigue reference value at a moment may be a value representing the fatigue level of the service provider at that moment.

[0122] Specifically, for each moment, the feature area image captured at that moment is input into the fatigue detection model, and a reference fatigue value representing the fatigue level of the service provider at that moment can be determined.

[0123] In step 1032, the fatigue degree of the service provider in a time period can be determined according to the reference fatigue value corresponding to each moment in the time period.

[0124] Specifically, the average value of the reference fatigue value in a time period can be calculated based on the reference fatigue value corresponding to each moment in the time period, and the calculated average value can be used to characterize the fatigue level of the service provider in the time period. The reference fatigue value at a moment cannot represent the fatigue level of the service provider in that period of time. The service provider may perform actions similar to fatigue (such as normal eye closure, coughing, etc.) at a certain moment due to necessary physiological reactions. It is inaccurate to determine the fatigue level of the service provider only because of the reference fatigue value determined at this time. Therefore, determining the fatigue level of the service provider through reference fatigue values ​​at multiple moments can reduce the inaccuracy of the determined fatigue level caused by abnormal reference fatigue values.

[0125] For example, the normal state of the service provider is represented by a value of 0, the mild fatigue state is represented by a value of 1, and the severe fatigue state is represented by a value of 2. All driving monitoring images of the service provider within 2 seconds are obtained, and the time interval for shooting driving monitoring images is 0.2 seconds, then a total of 10 driving monitoring images are obtained within 2 seconds, and the fatigue reference values ​​corresponding to the 10 consecutive driving monitoring images are: 2, 2, 0, 2, 2, 2, 1, 0, 2, 0. After calculation, the average reference fatigue value corresponding to the 10 consecutive driving monitoring images is 1.3, so the corresponding fatigue level of the service provider within 2 seconds is 1.3.

[0126] A driving monitoring image may contain two eyes and a mouth. Therefore, a plurality of facial feature points may be obtained in a driving monitoring image, and a plurality of regional feature images may be obtained based on the plurality of facial feature points. If the fatigue degree of the service provider is determined by only one regional feature image, there may be a deviation in the determined fatigue degree. For example, the eyes of a certain service provider are relatively large, and only the area where the eyes are located is used as the regional feature image. When the service provider yawns, the degree of opening and closing of the eyes does not decrease. At this time, the fatigue degree of the service provider determined based on the feature area image corresponding to the eyes is in a normal state, which deviates from the actual situation. Therefore, the fatigue degree of the service provider can be determined by a combination of multiple features. When multiple feature area images are intercepted, step S103 includes:

[0127] Step 1033, for each feature region image, input the feature region image into a corresponding fatigue detection model to determine a fatigue reference value corresponding to the feature region image;

[0128] Step 1034: If the fatigue reference value corresponding to at least one regional characteristic image exceeds a preset value, it is determined that the fatigue level of the service provider is in a fatigue state.

[0129] In step 1033, since the features corresponding to each feature area are different, there is a corresponding fatigue detection model for each feature area. Only when each feature area image is input into the fatigue detection model corresponding to the feature area image can the fatigue reference value corresponding to the feature area image be obtained.

[0130] For example, when the feature area picture is a feature area picture of the left eye, the model corresponding to the feature area picture is a left eye fatigue detection model; when the feature area picture is a feature area picture of the right eye, the model corresponding to the feature area picture is a right eye fatigue detection model; when the feature area picture is a feature area picture of the mouth, the model corresponding to the feature area picture is a mouth fatigue detection model.

[0131] In step 1032, the fatigue reference corresponding to each feature area image is compared with a preset value. When the fatigue references corresponding to all feature area images do not exceed the preset value, it means that the service provider is not in a fatigue state. When the fatigue reference corresponding to at least one feature area image exceeds the preset value, it means that the service provider is in a fatigue state. Because the fatigue level of the service provider is closely related to the personal safety of the service provider, as long as there is a feature area image that detects that the service provider is in a fatigue state, it can be considered that the service provider is in a fatigue state at this time. By jointly determining the fatigue level of the service provider using multiple different feature area images obtained in the driving monitoring image, the situation in which the fatigue of the service provider cannot be detected when the fatigue level of the service provider is determined by only one feature area image is reduced.

[0132] As can be seen from the above, the fatigue level may include a normal state, a mild fatigue state, and a severe fatigue state. Therefore, in this application, after the fatigue level of the service provider is determined by the fatigue detection model, different treatment methods for the service provider may be determined according to different fatigue levels, including:

[0133] Step 104: Determine a processing strategy based on the fatigue level of the service provider.

[0134] In step 104, the processing strategy may be a processing method for reducing the fatigue of the service provider, and the processing strategy may include one or more of the following processing methods: generating a prompt instruction for the service provider, and stopping the dispatch of service orders to the service provider. Among them, the prompt instruction can be used to generate a prompt action to prompt the service provider, and the prompt instruction can directly act on the client of the service provider (such as a mobile phone, a tablet computer, etc.), or can be sent to the service provider through the client of the service provider (when the client of the service provider is connected to the service provider through Bluetooth or a data cable, the prompt instruction can be sent to the service provider through the client of the service provider). The prompt instruction may include any one or more of the following instructions: voice prompt instruction, steering wheel shaking instruction and seat shaking instruction. The voice prompt instruction can control the client of the service provider to issue a voice prompt message, or control the sound playback device (car audio) of the service provider to issue a voice prompt message. The steering wheel shaking instruction can control the steering wheel of the service provider to shake without affecting normal driving. The seat shaking instruction can control the seat of the service provider (the seat where the service provider sits) to shake without affecting normal driving. A service order can be an order received by an online car-hailing service. A service order is initiated by a service requester (passenger) through a client. The service requester's client sends the service order to a server, which then dispatches the service order to the online car-hailing service. Stopping the dispatch of service orders to service providers can reduce the service provider's working hours, allowing them to have enough rest and reduce their fatigue.

[0135] Specifically, the fatigue level of the service provider can be divided into normal state, mild fatigue state and severe fatigue state. Different processing strategies can be used according to the actual fatigue level. Different processing strategies can improve the work efficiency of the service provider to varying degrees. When the fatigue level is normal, no processing can be performed on the service provider; when the fatigue level is mild fatigue, a more gentle processing method can be selected for the service provider, and the service provider can be reminded in time to wake up the service provider as soon as possible and improve work efficiency; when the fatigue level is severe fatigue, a more stringent processing method can be selected for the service provider, and the service provider can be directly allowed to rest, so that the service provider can work with higher efficiency after resting.

[0136] The fatigue level may be expressed in numerical form, therefore, step 104 includes:

[0137] Step 1041, when the fatigue level is greater than a first preset value and less than or equal to a second preset value, executing a first processing strategy; wherein the first processing strategy uses a non-compulsory processing method to allow the service provider to rest;

[0138] Step 1042, when the fatigue level is greater than a second preset value, execute a second processing strategy; wherein the first processing strategy uses a forced processing method to make the service provider take a rest.

[0139] Specifically, the fatigue level can be expressed in the form of a numerical value. When the fatigue level is less than or equal to a first preset value, the fatigue level is in a normal state; when the fatigue level is greater than the first preset value and less than or equal to a second preset value, the fatigue level is in a mild fatigue state; when the fatigue level is greater than the second preset value, the fatigue level is in a severe fatigue state. The first processing strategy and the second processing strategy have different processing methods. The first processing method is more moderate than the second processing method. The first processing strategy may include generating a prompt instruction for the service provider; the second processing strategy may include stopping the dispatch of service orders to the service provider.

[0140] In step 1041, when the fatigue level is greater than the first preset value and less than or equal to the second preset value, the fatigue level is a mild fatigue state. At this time, a more moderate processing method can be selected, that is, generating a prompt instruction for the service provider, so that the client of the service provider or the sound playback device of the service provider plays a voice message that can remind the service provider, or the steering wheel of the service provider or the seat where the service provider sits shakes to make the service provider more awake. The first processing strategy can make the service provider more awake without affecting the driving efficiency of the service provider, which will not reduce the driving efficiency of the service provider and can also improve the safety of the service provider.

[0141] In step 1042, when the fatigue level is greater than the second preset value, the fatigue level is a severe fatigue state. At this time, the service provider is extremely short of rest. Therefore, in order to increase the rest time of the service provider, the server can stop dispatching service orders to the service provider. The second processing strategy can allow the service provider to have enough rest time, reduce the situation of fatigue driving, and improve the safety of the service provider.

[0142] Although the characteristic area images corresponding to the service provider can be obtained in real time, the characteristic area images obtained are different due to the influence of lighting factors. In order to improve the accuracy of determining the fatigue degree of the service provider, corresponding fatigue detection models can be set for different light intensities, such as Figure 2 As shown, when the fatigue detection model includes multiple models, step S103 includes:

[0143] S201, inputting the feature area image into a strong light fatigue detection model and a dark light fatigue detection model respectively to obtain output results of each fatigue detection model;

[0144] S202: Perform weighted summation based on the output results of each fatigue detection model and the weight value corresponding to each output result to obtain the fatigue level of the service provider.

[0145] In step S201, the strong light fatigue detection model and the dark light fatigue detection model can be distinguished according to the different light intensities. The strong light fatigue detection model is obtained by training based on driving monitoring images under strong light, and the dark light fatigue detection model is obtained by training based on driving monitoring images under dark light. In addition to the strong light fatigue detection model and the dark light fatigue detection model described above, the fatigue detection model can also include a comprehensive illumination fatigue detection model, which is obtained by training based on driving monitoring images under strong light and driving monitoring images under dark light. The output result can be a probability value that characterizes each fatigue state, and the fatigue state can include the normal state, mild fatigue state, and severe fatigue state mentioned above.

[0146] Specifically, after a regional feature image is input into a plurality of different fatigue detection models, each fatigue detection model will output an output result corresponding to the regional feature image.

[0147] For example, there are three fatigue detection models, namely, strong light fatigue detection model, dim light fatigue detection model and comprehensive light fatigue detection model. There is a feature area image of a mouth. After the feature area image is input into the three fatigue detection models respectively, each fatigue detection model will obtain the probability values ​​corresponding to the normal state, mild fatigue state and severe fatigue state, that is, the output results of the strong light fatigue detection model are 0.3, 0.5 and 0.2 respectively, the output results of the dim light fatigue detection model are 0.1, 0.6 and 0.3 respectively, and the output results of the comprehensive light fatigue detection model are 0.15, 0.7 and 0.15 respectively.

[0148] In step S202, the weight value corresponding to each output result can represent the importance of the fatigue detection model corresponding to each output result. The higher the weight value, the more important the fatigue detection model is, and the lower the weight value, the less important the fatigue detection model is. Since the light intensity can change with time, the weight value corresponding to each output result can be determined based on the current moment; or, when training the fatigue detection model, the number of samples corresponding to each training model is different, so the weight value corresponding to each output result can be determined based on the amount of training sample data of the fatigue detection model corresponding to the output result.

[0149] Specifically, in order to make the determined fatigue level of the service provider more accurate, it is necessary to determine the fatigue level of the service provider based on the weight value corresponding to each fatigue detection model and the output result corresponding to each fatigue detection model. That is, the output result of each fatigue detection model is multiplied by the corresponding weight value, and the corresponding fatigue state results are added, and the maximum value is determined from the added results. The maximum value or the fatigue state corresponding to the maximum value is the fatigue level of the service provider. Calculating the fatigue level through the results of the fatigue detection model output corresponding to different light intensities can reduce the influence of light intensity on the fatigue level and improve the accuracy of determining the fatigue level of the service provider.

[0150] Continuing the output results of the above three fatigue detection models, the weight values ​​corresponding to the strong light fatigue detection model, the dark light fatigue detection model and the comprehensive light fatigue detection model are 0.25, 0.25 and 0.5 respectively. The fatigue level corresponding to the normal state is 0.25*0.3+0.25*0.1+0.5*0.15=1.075, the fatigue level corresponding to the mild fatigue state is 0.25*0.5+0.25*0.6+0.5*0.7=1.925, and the fatigue level corresponding to the severe fatigue state is 0.25*0.2+0.25*0.3+0.5*0.15=1.25. The value corresponding to the mild fatigue state is the largest, and it is determined that the current service provider is in a mild fatigue state.

[0151] The present application provides a vehicle-mounted device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the vehicle-mounted device is running, the steps of the fatigue driving detection method mentioned above are executed. The vehicle-mounted device can be a device installed on the service provider end to obtain driving monitoring images. The vehicle-mounted device can be installed on the rearview mirror inside the vehicle, installed at the air-conditioning outlet on the left side of the steering wheel, installed at the air-conditioning outlet on the right side of the steering wheel, installed on the roof diagonally above the service provider, etc. The installation position of the vehicle-mounted device can be any position that can obtain the facial image of the service provider and does not hinder the normal driving of the service provider. The vehicle-mounted device can include a vehicle recorder, a camera, a mobile phone, a tablet computer, and other devices that can obtain images.

[0152] like Figure 3As shown, a block diagram of a service system 100 in which a method for detecting fatigue driving in some embodiments of the present application is located is shown. For example, the service system 100 may be an online transportation service platform for transportation services such as taxis, chauffeur services, express cars, carpooling, bus services, driver rentals, or shuttle services, or any combination thereof. The service system 100 may include one or more of a server 110 (a type of execution subject of the method provided in the present application), a network 120, a service request end 130 (service requester), a service provider end 140 (service provider), and a database 150, and the server 110 may include a processor for executing instruction operations.

[0153] In some embodiments, the server 110 may be a single server or a server group. The server group may be centralized or distributed (for example, the server 110 may be a distributed system). In some embodiments, the server 110 may be local or remote relative to the terminal. For example, the server 110 may access information and / or data stored in the service request end 130, the service provider end 140, or the database 150, or any combination thereof, via the network 120. As another example, the server 110 may be directly connected to at least one of the service request end 130, the service provider end 140, and the database 150 to access the stored information and / or data. In some embodiments, the server 110 may be implemented on a cloud platform; as an example only, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, a multi-cloud, etc., or any combination thereof. In some embodiments, the server 110 may be implemented on a cloud platform having the cloud computing capabilities described in this application. Figure 3 One or more components shown are implemented on the electronic device 1000 .

[0154] The network 120 can be used for the exchange of information and / or data. In some embodiments, one or more components in the service system 100 (e.g., the server 110, the service request end 130, the service provider 140, and the database 150) can send information and / or data to other components. For example, the server 110 can obtain a service request from the service request end 130 via the network 120. In some embodiments, the network 120 can be any type of wired or wireless network, or a combination thereof. As an example only, the network 120 can include a wired network, a wireless network, an optical fiber network, a telecommunication network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a wide area network (WAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, or a near field communication (NFC) network, or any combination thereof. In some embodiments, network 120 may include one or more network access points. For example, network 120 may include a wired or wireless network access point, such as a base station and / or a network switching node, through which one or more components of service system 100 may connect to network 120 to exchange data and / or information.

[0155] In some embodiments, the service requester of the service requesting end 130 may be someone other than the actual service demander. For example, the service requester A of the service requesting end 130 may use the service requesting end 130 to initiate a service request for the actual service demander B (for example, the service requester A may call a car for his friend B), or receive service information or instructions from the server 110. In some embodiments, the service requester of the service providing end 140 may be the actual service provider, or someone other than the actual service provider. For example, the service requester C of the service providing end 140 may use the service providing end 140 to receive a service request provided by the actual service provider D (for example, the service requester C may receive an order from the service provider D hired by himself), and / or information or instructions from the server 110. In some embodiments, "service requester" and "service requesting end" may be used interchangeably, and "service provider" and "service providing end" may be used interchangeably.

[0156] In some embodiments, the service request end 130 may include a mobile device, a tablet computer, a laptop computer, or a built-in device in a motor vehicle, or any combination thereof. In some embodiments, the mobile device may include a smart home device, a wearable device, a smart mobile device, a virtual reality device, or an augmented reality device, or any combination thereof. In some embodiments, the smart home device may include a smart lighting device, a control device for a smart electrical device, a smart monitoring device, a smart TV, a smart camera, or an intercom, or any combination thereof. In some embodiments, the wearable device may include a smart bracelet, a smart shoelace, a smart glass, a smart helmet, a smart watch, smart clothing, a smart backpack, a smart accessory, or any combination thereof. In some embodiments, the smart mobile device may include a smart phone, a personal digital assistant (PDA), a gaming device, a navigation device, or a point of sale (POS) device, or any combination thereof. In some embodiments, the virtual reality device and / or the augmented reality device may include a virtual reality helmet, a virtual reality glass, a virtual reality patch, an augmented reality helmet, an augmented reality glass, or an augmented reality patch, or any combination thereof. For example, the virtual reality device and / or the augmented reality device may include various virtual reality products, etc. In some embodiments, the built-in device in the motor vehicle may include an onboard computer, an onboard TV, etc. In some embodiments, the service requester 130 may be a device having a positioning technology for locating the position of the service requester and / or the service requester.

[0157] In some embodiments, the service provider 140 may be a device similar to or identical to the service requester 130. In some embodiments, the service provider 140 may be a device with positioning technology for locating the location of the service provider and / or the service provider. In some embodiments, the service requester 130 and / or the service provider 140 may communicate with other positioning devices to determine the location of the service requester, the service requester 130, the service provider, or the service provider 140, or any combination thereof. In some embodiments, the service requester 130 and / or the service provider 140 may send positioning information to the server 110.

[0158] The database 150 may store data and / or instructions. In some embodiments, the database 150 may store data obtained from the service requester 130 and / or the service provider 140. In some embodiments, the database 150 may store data and / or instructions of the exemplary methods described in the present application. In some embodiments, the database 150 may include a mass storage device, a removable storage device, a volatile read-write memory, or a read-only memory (ROM), etc., or any combination thereof. As an example, mass storage may include magnetic disks, optical disks, solid-state drives, etc.; removable storage may include flash drives, floppy disks, optical disks, memory cards, zip disks, magnetic tapes, etc.; volatile read-write storage may include random access memory (Random Access Memory, RAM); RAM may include dynamic RAM (Dynamic Random Access Memory, DRAM), double data rate synchronous dynamic RAM (Double Date-Rate Synchronous RAM, DDRSDRAM); static RAM (Static Random-Access Memory, SRAM), thyristor RAM (Thyristor-Based Random Access Memory, T-RAM) and zero capacitor RAM (Zero-RAM), etc. As an example, ROM may include mask ROM (Mask Read-Only Memory, MROM), programmable ROM (Programmable Read-Only Memory, PROM), erasable programmable ROM (Programmable Erasable Read-only Memory, PEROM), electrically erasable programmable ROM (Electrically Erasable Programmable read only memory, EEPROM), CD-ROM (CD-ROM), and digital universal disk ROM, etc. In some embodiments, database 150 can be implemented on a cloud platform. As an example only, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, a cross-cloud, a multi-cloud, or other similar, etc., or any combination thereof.

[0159] Based on the same inventive concept, an embodiment of the present application also provides a device for determining the unlocking progress corresponding to the above-mentioned fatigue driving detection method. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned fatigue driving detection method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0160] like Figure 4As shown, the embodiment of the present application provides a fatigue driving detection device, comprising:

[0161] A determination module 401 is used to input the acquired driving monitoring image of the service provider into a facial feature point detection model to determine facial feature points;

[0162] The interception module 402 is used to intercept a feature area image from the driving monitoring image according to the position of the facial feature points;

[0163] The fatigue level determination module 403 is used to input the feature area image into the fatigue detection model to obtain the fatigue level of the service provider.

[0164] In some embodiments, the determination module 401 includes:

[0165] A first determination subunit, for each moment, is used to input the driving monitoring image of the service provider at that moment into the facial feature determination model to obtain facial feature points corresponding to that moment;

[0166] The interception module 402 includes:

[0167] The first interception subunit is used to intercept, for each moment, a feature area image at that moment from the driving monitoring image at that moment according to the corresponding facial feature points.

[0168] In some embodiments, the fatigue level determination module 403 includes:

[0169] A first reference fatigue value determination unit, for each moment, is used to input the intercepted feature area image into the fatigue detection model to obtain a reference fatigue value at that moment;

[0170] The first fatigue level determination subunit is used to determine the fatigue level of the service provider according to the reference fatigue value at each moment.

[0171] In some embodiments, the facial landmarks include one or more of the following locations:

[0172] Left eye feature points, right eye feature points and mouth feature points;

[0173] The left eye feature points include: the position of the center point of the left eye, the position of the tail of the left eye, and the position of the corner of the left eye;

[0174] The right eye feature points include: the position of the center point of the right eye, the position of the end of the right eye, and the position of the corner of the right eye;

[0175] The mouth feature points include: the position of the mouth center point, the position of the left corner of the mouth, and the position of the right corner of the mouth.

[0176] In some embodiments, the feature area image includes any one or more of the following images:

[0177] Left eye area image, right eye area image and mouth area image.

[0178] In some embodiments, when multiple characteristic region images are captured, the fatigue level determination module 403 includes:

[0179] A second reference fatigue value determination unit, for each feature region image, is used to input the feature region image into a corresponding fatigue detection model to determine a fatigue reference value corresponding to the feature region image;

[0180] The second fatigue level determination subunit is configured to determine that the fatigue level of the service provider is a fatigue state if the fatigue reference value corresponding to at least one regional characteristic image exceeds a preset value.

[0181] In some embodiments, it also includes:

[0182] The processing module is used to determine the processing strategy according to the fatigue level of the service provider.

[0183] In some embodiments, the processing strategy includes one or more of the following processing methods:

[0184] Generate prompt instructions for the service provider and stop dispatching service orders to the service provider.

[0185] In some embodiments, the prompt instruction includes any one or more of the following instructions:

[0186] Voice prompt commands, steering wheel vibration commands and seat vibration commands.

[0187] In some embodiments, the processing module includes:

[0188] A first processing subunit is configured to execute a first processing strategy when the fatigue level is greater than a first preset value and the fatigue level is less than or equal to a second preset value; wherein the first processing strategy uses a non-compulsory processing method to allow the service provider to rest;

[0189] The second processing subunit is used to execute a second processing strategy when the fatigue level is greater than a second preset value; wherein the first processing strategy uses a forced processing method to make the service provider take a rest.

[0190] In some embodiments, the first processing strategy includes generating a prompt instruction for the service provider;

[0191] The second handling strategy includes stopping dispatching service orders to the service provider.

[0192] In some embodiments, when the fatigue detection model includes multiple models, the fatigue level determination module 403 includes:

[0193] An output result unit, used for inputting the feature area image into the strong light fatigue detection model and the dark light fatigue detection model respectively, so as to obtain the output result of each fatigue detection model;

[0194] The third fatigue level determination subunit is used to perform weighted summation according to the output results of each fatigue detection model and the weight value corresponding to each output result, so as to obtain the fatigue level of the service provider.

[0195] In some embodiments, the weight value corresponding to each output result is determined based on the current moment; or, the weight value corresponding to each output result is determined based on the amount of training sample data of the fatigue detection model corresponding to the output result.

[0196] Corresponding to the above method, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the fatigue driving detection method are executed.

[0197] like Figure 5 As shown, it is a schematic diagram of an electronic device provided in an embodiment of the present application. The electronic device 1000 includes: a processor 1001, a memory 1002 and a bus 1003. The memory 1002 stores execution instructions. When the electronic device is running, the processor 1001 communicates with the memory 1002 through the bus 1003, and the processor 1001 executes the steps of the fatigue driving detection method stored in the memory 1002.

[0198] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the above modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0199] The modules described above as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0200] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0201] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the above methods of each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0202] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for detecting fatigue driving, characterized in that: include: For each moment, inputting the acquired driving monitoring image of the service provider at that moment into the facial feature determination model to determine facial feature points; According to the position of the facial feature points corresponding to each moment, extracting a feature area image at that moment from the driving monitoring image at that moment; For each of the feature area images captured at each moment, the feature area image is input into a corresponding plurality of fatigue detection models, so as to determine a reference fatigue value corresponding to the feature area image based on an output result of each fatigue detection model and a weight value of the fatigue detection model, wherein the plurality of fatigue detection models correspond to different light intensities; as well as The fatigue degree of the service provider is determined according to the reference fatigue value of each characteristic region image at each moment.

2. The method according to claim 1, characterized in that The facial feature points include one or more of the following positions: Left eye feature points, right eye feature points and mouth feature points; The left eye feature points include: the position of the center point of the left eye, the position of the tail of the left eye and the position of the corner of the left eye; The right eye feature points include: the position of the center point of the right eye, the position of the end of the right eye and the position of the corner of the right eye; The mouth feature points include: the position of the mouth center point, the position of the left mouth corner and the position of the right mouth corner.

3. The method according to claim 1, characterized in that The characteristic area image includes any one or more of the following images: Left eye area image, right eye area image and mouth area image.

4. The method according to claim 1, characterized in that When a plurality of feature area images are captured, the feature area images captured at each moment are input into a fatigue detection model corresponding to the feature area images to obtain a reference fatigue value corresponding to the feature area images, and the fatigue degree of the service provider is determined according to the reference fatigue value of each feature area image at each moment, including: For each feature region image, input the feature region image into a corresponding fatigue detection model to determine a reference fatigue value corresponding to the feature region image; If the reference fatigue value corresponding to at least one regional characteristic image exceeds a preset value, it is determined that the fatigue level of the service provider is in a fatigue state.

5. The method according to claim 1, characterized in that Also includes: A processing strategy is determined according to the fatigue level of the service provider.

6. The method according to claim 5, characterized in that The processing strategy includes one or more of the following processing methods: Generate a prompt instruction for the service provider and stop dispatching service orders to the service provider.

7. The method according to claim 6, characterized in that The prompt instruction includes any one or more of the following instructions: Voice prompt commands, steering wheel vibration commands and seat vibration commands.

8. The method according to claim 5, characterized in that Determine a handling strategy based on the fatigue level of the service provider, including: When the fatigue level is greater than a first preset value, and the fatigue level is less than or equal to a second preset value, executing a first processing strategy; wherein the first processing strategy uses a non-compulsory processing method to allow the service provider to rest; When the fatigue level is greater than a second preset value, a second processing strategy is executed; wherein the first processing strategy uses a forced processing method to make the service provider take a rest.

9. The method according to claim 8, characterized in that The first processing strategy includes generating a prompt instruction for the service provider; The second processing strategy includes stopping dispatching service orders to the service provider.

10. The method according to claim 1, characterized in that When the fatigue detection model includes multiple models, the captured feature area image at each moment is input into the fatigue detection model corresponding to the feature area image to obtain a reference fatigue value corresponding to the feature area image, and the fatigue degree of the service provider is determined according to the reference fatigue value at each moment, including: Inputting the characteristic region image into a strong light fatigue detection model and a dark light fatigue detection model respectively to obtain output results of each fatigue detection model; A weighted sum is performed according to the output result of each fatigue detection model and the weight value corresponding to each output result to obtain the fatigue degree of the service provider.

11. The method according to claim 9, characterized in that The weight value corresponding to each output result is determined according to the current moment; or, the weight value corresponding to each output result is determined according to the amount of training sample data of the fatigue detection model corresponding to the output result.

12. A fatigue driving detection device, characterized in that: include: A determination module, for inputting, at each moment, the acquired driving monitoring image of the service provider at that moment into a facial feature determination model to determine facial feature points; A capture module, used for capturing a feature area image at each moment from the driving monitoring image at that moment according to the position of the facial feature points corresponding to that moment; For each of the feature area images captured at each moment, the feature area image is input into a corresponding plurality of fatigue detection models, so as to determine a reference fatigue value corresponding to the feature area image based on an output result of each fatigue detection model and a weight value of the fatigue detection model, wherein the plurality of fatigue detection models correspond to different light intensities; A fatigue degree determination module is used to input the feature area image corresponding to each moment into a fatigue detection model corresponding to the feature area image to obtain a reference fatigue value corresponding to the feature area image, and determine the fatigue degree of the service provider according to the reference fatigue value of each feature area image at each moment.

13. A vehicle-mounted device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

14. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of any method as claimed in claims 1 to 11.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are executed.

Images