Driving state monitoring method and apparatus, device, storage medium and product

Abstract

A driving state monitoring method, belonging to the technical field of vehicles. The driving state monitoring method comprises: extracting face features of a driver from a face image of the driver; sending the face features to a cloud server, wherein the cloud server determines a target account corresponding to the face features, and acquires account data of the target account, the account data comprising warning configuration information corresponding to abnormal driving states; determining a fatigue state of the driver according to the face image; when the driver is in fatigue driving, acquiring a warning mode corresponding to the fatigue level from the warning configuration information corresponding to the abnormal driving states; and, by means of the warning mode, warning the driver, thus ensuring the driving safety of users, and reducing the occurrence rate of traffic accidents. Further disclosed are a driving state monitoring apparatus, a control device, a computer-readable storage medium and a computer program product.

Description

Driving state monitoring method, device, equipment, storage medium and product

[0001] This application claims priority to Chinese patent application number 202310789333.6, filed on June 29, 2023, entitled “Driving state monitoring method, device, equipment and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of vehicle technology, and in particular to a driving status monitoring method, device, equipment, storage medium and product. Background Art

[0003] As vehicle usage continues to increase, the incidence of traffic accidents is also increasing. Among them, a very important factor in traffic accidents is driving fatigue. When users drive a vehicle for a long time, they may experience mental fatigue, limb weakness, lack of concentration, and decreased judgment. These conditions will affect the user's safety during the driving process and cause traffic accidents. Therefore, how to ensure the user's driving safety and thus reduce the incidence of traffic accidents has become an urgent problem to be solved in this field.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a driving status monitoring method, device, equipment, storage medium, and product that can promptly remind users when they are driving fatigued, thereby ensuring the user's driving safety and reducing the incidence of traffic accidents. The technical solution is as follows:

[0006] In one aspect, a driving state monitoring method is provided, the method comprising:

[0007] Acquire a facial image of the driver of the vehicle;

[0008] extracting facial features of the driver based on the facial image and sending the facial features to a cloud server; wherein the cloud server is configured to determine a target account of the driver based on the facial features and obtain account data of the target account, wherein the account data includes reminder configuration information corresponding to abnormal driving status;

[0009] Receiving the account data of the target account sent by the cloud server;

[0010] determining, based on the facial image, a frequency and duration of eye closure of the driver within a first preset time period;

[0011] determining the driver's fatigue information based on the eye-closing frequency and the eye-closing duration, the fatigue information including a fatigue state and a fatigue level corresponding to the fatigue state;

[0012] In a case where the fatigue state is fatigue, obtaining a first correspondence between a fatigue level and a reminder mode from reminder configuration information corresponding to the abnormal driving state;

[0013] Based on the fatigue level corresponding to the fatigue state, determining a first reminder method corresponding to the fatigue level from the first corresponding relationship;

[0014] The driver is reminded based on the first reminder method.

[0015] In a possible implementation, determining the driver's fatigue information based on the eye-closing frequency and the eye-closing duration includes:

[0016] When the eye-closing frequency reaches a first frequency and the eye-closing duration is greater than a first duration but less than a second duration, determining that the driver's fatigue state is fatigue and the fatigue level is a first level;

[0017] When the eye-closing frequency reaches a second frequency and the eye-closing duration is greater than the second duration but less than a third duration, determining that the driver's fatigue state is fatigue and the fatigue level is a second level;

[0018] When the eye-closing frequency reaches the third frequency and the eye-closing duration is greater than the third duration, it is determined that the driver's fatigue state is fatigue and the fatigue level is the third level; wherein, the third frequency is less than the second frequency, the second frequency is less than the first frequency, and the fatigue degrees corresponding to the first level, the second level and the third level increase respectively.

[0019] In another possible implementation, the method further includes:

[0020] Determining an actual gaze area of ​​the driver based on the facial image;

[0021] If the actual gaze area does not match the theoretical gaze area, determining that the driver is in a distracted state;

[0022] Obtaining a second correspondence between the abnormal state and the reminder method from the reminder configuration information corresponding to the abnormal driving state;

[0023] Obtaining a second reminder method corresponding to the distracted state from the second corresponding relationship;

[0024] The driver is reminded based on the second reminder method.

[0025] In another possible implementation, the method further includes:

[0026] detecting the driver's hands based on the facial image;

[0027] When detecting that the driver's hand is holding the electronic device, determining the distance between the driver's hand and the ear;

[0028] When the distance is less than a preset distance, determining that the driver is in a talking state;

[0029] Obtaining a third reminder method corresponding to the call status from the second corresponding relationship;

[0030] The driver is reminded based on the third reminder method.

[0031] In another possible implementation, the method further includes:

[0032] In the case of detecting that the driver's hand is holding a smoking object, detecting the driver's mouth;

[0033] determining that the driver is smoking when the smoking object is detected to be in the driver's mouth;

[0034] obtaining a smoke concentration in the vehicle, and determining an environment in which the vehicle is located if the smoke concentration is greater than a preset concentration;

[0035] Determining a ventilation method based on the environment in which the vehicle is located;

[0036] The vehicle is ventilated based on the ventilation method.

[0037] In another possible implementation, the account data of the target account further includes the identity information of the driver; and the method further includes:

[0038] determining the driver's emotion type based on the facial features;

[0039] When the driver's emotion type is positive, obtaining the driver's identity information from the account data of the target account;

[0040] Based on the positive emotion and the identity information, a voice message is broadcasted or music is recommended to the driver.

[0041] In another possible implementation, the account data of the target account further includes vehicle body configuration data; and the method further includes:

[0042] Acquiring position information of a driver's seat and position information of a rearview mirror from the vehicle body configuration data;

[0043] Based on the position information of the main driver's seat and the position information of the rearview mirror, the position of the main driver's seat and the position of the rearview mirror are adjusted.

[0044] In another aspect, a driving state monitoring device is provided, the device comprising:

[0045] A first acquisition module is used to acquire a facial image of the driver of the vehicle;

[0046] an extraction module configured to extract facial features of the driver based on the facial image and transmit the facial features to a cloud server; wherein the cloud server is configured to determine a target account of the driver based on the facial features and obtain account data of the target account, wherein the account data includes reminder configuration information corresponding to abnormal driving conditions;

[0047] a receiving module, configured to receive the account data of the target account sent by the cloud server;

[0048] a first determining module, configured to determine, based on the facial image, a frequency and duration of eye closure of the driver within a first preset time period;

[0049] a second determining module, configured to determine fatigue information of the driver based on the eye-closing frequency and the eye-closing duration, the fatigue information including a fatigue state and a fatigue level corresponding to the fatigue state;

[0050] a second acquisition module, configured to acquire, when the fatigue state is fatigue, a first correspondence between a fatigue level and a reminder method from reminder configuration information corresponding to the abnormal driving state;

[0051] a third determining module, configured to determine, based on the fatigue level corresponding to the fatigue state, a first reminder method corresponding to the fatigue level from the first corresponding relationship;

[0052] A reminder module is used to remind the driver based on the first reminder method.

[0053] In one possible implementation, the second determination module is used to determine that the driver's fatigue state is fatigue and the fatigue level is the first level when the eye-closing frequency reaches a first frequency and the eye-closing duration is greater than the first duration but less than the second duration; to determine that the driver's fatigue state is fatigue and the fatigue level is the second level when the eye-closing frequency reaches a second frequency and the eye-closing duration is greater than the second duration but less than a third duration; and to determine that the driver's fatigue state is fatigue and the fatigue level is the third level when the eye-closing frequency reaches a third frequency and the eye-closing duration is greater than the third duration; wherein the third frequency is less than the second frequency, the second frequency is less than the first frequency, and the fatigue degrees corresponding to the first level, the second level and the third level respectively increase in sequence.

[0054] In another possible implementation, the apparatus further includes:

[0055] A fourth determining module, configured to determine an actual gaze area of ​​the driver based on the facial image;

[0056] a fifth determining module, configured to determine that the driver is in a distracted state if the actual gaze area does not match the theoretical gaze area;

[0057] a third acquisition module, configured to acquire a second correspondence between the abnormal state and the reminder method from the reminder configuration information corresponding to the abnormal driving state;

[0058] a fourth acquiring module, configured to acquire a second reminder method corresponding to the distracted state from the second corresponding relationship;

[0059] The reminding module is further configured to remind the driver based on the second reminding method.

[0060] In another possible implementation, the apparatus further includes:

[0061] a first detection module, configured to detect the driver's hands based on the facial image;

[0062] a sixth determining module, configured to determine a distance between the driver's hand and the ear when detecting that the driver's hand is holding the electronic device;

[0063] a seventh determining module, configured to determine that the driver is in a talking state when the distance is less than a preset distance;

[0064] A fifth obtaining module, configured to obtain a third reminder method corresponding to the call status from the second corresponding relationship;

[0065] The reminding module is further configured to remind the driver based on the third reminding method.

[0066] In another possible implementation, the apparatus further includes:

[0067] a second detection module, configured to detect the driver's mouth when detecting that the driver's hand is holding a smoking object;

[0068] an eighth determining module, configured to determine that the driver is smoking when detecting that the smoking object is contained in the driver's mouth;

[0069] an eighth determining module, configured to obtain a smoke concentration in the vehicle, and determine an environment in which the vehicle is located if the smoke concentration is greater than a preset concentration;

[0070] a ninth determining module, configured to determine a ventilation mode based on an environment in which the vehicle is located;

[0071] A ventilation module is used to ventilate the vehicle based on the ventilation method.

[0072] In another possible implementation, the account data of the target account further includes the identity information of the driver; and the device further includes:

[0073] a tenth determining module, configured to determine the driver's emotion type based on the facial features;

[0074] a sixth acquisition module, configured to acquire the driver's identity information from the account data of the target account when the driver's emotion type is positive;

[0075] A recommendation module is used to broadcast a voice message or recommend music to the driver based on the positive emotion and the identity information.

[0076] In another possible implementation, the account data of the target account further includes vehicle body configuration data; and the device further includes:

[0077] a seventh acquisition module, configured to acquire position information of a driver's seat and position information of a rearview mirror from the vehicle body configuration data;

[0078] An adjustment module is used to adjust the position of the main driver's seat and the position of the rearview mirror based on the position information of the main driver's seat and the position information of the rearview mirror.

[0079] On the other hand, a control device is provided, which includes a processor and a memory, wherein at least one program code is stored in the memory, and the at least one program code is loaded and executed by the processor to implement any of the above-mentioned driving status monitoring methods.

[0080] On the other hand, a computer-readable storage medium is provided, in which at least one program code is stored. The at least one program code is loaded and executed by a processor to implement any of the above-mentioned driving state monitoring methods.

[0081] On the other hand, a computer program product is provided, wherein at least one program code is stored in the computer program product, and the at least one program code is loaded and executed by a processor to implement any of the above-mentioned driving state monitoring methods.

[0082] The present application provides a driving status monitoring method. When a driver is driving a vehicle, the method uses facial recognition technology to obtain the driver's target account data from a cloud server. This account data includes reminder configuration information corresponding to abnormal driving conditions, thereby obtaining a reminder method suitable for the driver's abnormal driving conditions. Furthermore, when facial recognition technology identifies driver fatigue, the method uses the reminder configuration information corresponding to the driver's abnormal condition to obtain a reminder method suitable for the user in the fatigue state, thereby reminding the driver. This can improve the effectiveness of the reminder, thereby ensuring safe driving for the user and reducing the incidence of traffic accidents.

[0083] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] FIG1 is a schematic diagram of an implementation environment of a driving state monitoring method provided in an embodiment of the present application;

[0085] FIG2 is a flow chart of a driving state monitoring method provided in an embodiment of the present application;

[0086] FIG3 is a schematic diagram of an interaction between a control device and a cloud server provided in an embodiment of the present application;

[0087] FIG4 is a schematic diagram of a control device performing data processing through a DMS according to an embodiment of the present application;

[0088] FIG5 is a schematic structural diagram of a driving state monitoring device provided in an embodiment of the present application;

[0089] FIG6 is a structural block diagram of a control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0090] In order to make the technical solutions and advantages of the present application clearer, the implementation methods of the present application are described in further detail below.

[0091] The terms "first," "second," "third," and "fourth," etc. in the specification and claims of this application and the accompanying drawings are used to distinguish different objects, not to describe a specific order. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0092] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the facial images, emotion types, account data, etc. involved in this application are all obtained with full authorization.

[0093] Figure 1 is a schematic diagram of an implementation environment of a driving state monitoring method provided in an embodiment of the present application. The implementation environment includes: a camera module 10, a control device 11, a cloud server 12, and a T-BOX 13 (Telematics BOX, information processor). The camera module 10, the control device 11, and the T-BOX 13 are located in the same vehicle, which can be a fuel vehicle, an electric vehicle, or a hybrid vehicle, without specific limitation.

[0094] In an embodiment of the present application, the camera module 10 can capture the driver's facial image and send the facial image to the control device 11. The control device 11 extracts facial features from the facial image and sends the facial features to the T-BOX 13. The T-BOX 13 forwards the facial features to the cloud server 12. The cloud server 12 compares the facial features with the facial features stored in the cloud database, determines the target account corresponding to the facial features, obtains the account data of the target account, and sends the account data of the target account to the control device 11 through the T-BOX 13. The control device 11 receives and locally stores the account data of the target account. The target account is the account registered by the driver in the cloud server, and the account data of the target account is the relevant data associated with the target account; for example, the account data of the target account includes reminder configuration information corresponding to an abnormal driving state. The abnormal driving state refers to a driving state in which a traffic accident may occur. The reminder configuration information corresponding to the abnormal driving state includes a first correspondence between a fatigue level and a reminder method. The reminder configuration information corresponding to the abnormal driving state also includes a second correspondence between the abnormal state and the reminder method. The account data of the target account also includes the identity information of the driver. The account data of the target account also includes position information of the main driver's seat and position information of the rearview mirror.

[0095] In addition, after the control device 11 obtains the facial image, it can determine the driver's driving status based on the facial image, which includes: fatigue state, distraction state, call state, smoking state, etc. According to the driver's driving status, the corresponding reminder method is obtained from the account data of the target account, and then the driver is reminded based on the obtained reminder method.

[0096] The control device 11 can be a vehicle computer or vehicle controller. "Vehicle computer" is short for an in-vehicle infotainment system, including a host computer and a display screen. The cloud server 12 can be at least one of a single server, a server cluster consisting of multiple servers, a cloud computing platform, and a virtualization center. The camera module 10 is a camera that captures the driver in the driver's seat. The T-BOX 13 provides network transmission capabilities, enabling data transmission between the control device 11 and the cloud server 12.

[0097] In the embodiment of the present application, the control device can monitor the driver's fatigue state, distraction state, talking state and smoking state, and remind the driver through the reminder method corresponding to the monitored state. The following first introduces the monitoring process of the fatigue state.

[0098] FIG2 is a flow chart of a driving state monitoring method provided by an embodiment of the present application, which is executed by a control device. Referring to FIG2 , the method includes:

[0099] Step 201: The control device obtains a facial image of the driver of the vehicle.

[0100] The driver is the person who drives the vehicle, that is, the person sitting in the main driver's seat.

[0101] In this step, the camera module can collect facial images in real time or periodically. When a facial image is collected, the collected facial image is sent to the control device. Correspondingly, the control device receives the facial image sent by the camera module.

[0102] When the camera module is turned on, it can directly capture images. When the camera module is turned off, the control device can display a status monitoring option on the display screen. In response to detecting a trigger operation of the status monitoring option, the control device sends a start instruction to the camera module, which starts up and then captures facial images in real time or periodically.

[0103] In an embodiment of the present application, after the control device acquires a facial image, it determines the clarity and completeness of the facial image. If the clarity or completeness does not meet the requirements, a prompt message is displayed or voice broadcast to remind the user to adjust their posture. The camera module then retakes the driver's facial image and then determines whether the clarity and completeness of the retaken facial image meet the requirements. This process continues until a facial image with clarity and completeness that meets the requirements is captured, thereby allowing the camera module to capture a facial image that meets the requirements. If the clarity and completeness meet the requirements, the subsequent steps are executed.

[0104] Step 202: The control device extracts the driver's facial features based on the facial image and sends the facial features to the cloud server.

[0105] The control device extracts facial features from the facial image and sends them to the cloud server via T-BOX. The cloud server compares these facial features with multiple facial features stored in a cloud database and searches for a target facial feature that matches the facial features. If the cloud server finds a matching target facial feature, it determines the target account corresponding to the target facial feature based on the correspondence between facial features and accounts. It then retrieves the target account's account data from the cloud database and sends the target account's account data to the control device via T-BOX. This process is shown in Figure 3.

[0106] The target account is the driver's account registered in the cloud server, and the target account's account data is the relevant data associated with the target account. For example, the target account's account data includes reminder configuration information corresponding to abnormal driving conditions, which refers to driving conditions that may result in a traffic accident. The reminder configuration information corresponding to abnormal driving conditions includes a first correspondence between fatigue level and reminder method, and a second correspondence between abnormal conditions and reminder methods. The target account's account data also includes the driver's identity information, including the driver's gender and age. The target account's account data also includes vehicle configuration data, including the position of the driver's seat and rearview mirrors, steering wheel height, and other personalized driving habit settings, such as the brightness of the interior ambient light, music style, HUD (Head Up Display), ADAS (Advanced Driving Assistance System), and head unit, etc., without specific limitation. The head unit can provide users with functions such as Bluetooth, mobile phone interconnection, navigation, voice, phone calls, music, video, games, lifestyle services, and mini-programs.

[0107] The control device can extract facial features from the facial image itself or pass the facial image to the DMS (Driver Monitor System). The DMS then transmits the facial image to the visual perception service module via ZMQ communication. The visual perception service module extracts facial features from the facial image and returns them to the DMS, which then passes them to the control device. The control device then transmits them to the cloud server via T-BOX. Furthermore, the DMS can obtain vehicle body configuration data and pass it to the visual perception service module for processing. This data is CAN (Controller Area Network) data (see Figure 4).

[0108] For example, please continue to refer to Figure 3. If the cloud server does not find the target facial feature that matches the facial feature, the cloud server returns a first notification message to the control device through T-BOX. The control device displays a registration interface based on the first notification message, and registers the account data of the target account in the cloud server based on the registration interface.

[0109] The process of registering the target account's account data in the cloud server based on the registration interface by the control device may include: the driver triggering the registration interface displayed on the control device, the registration interface including multiple registration options, and the driver entering registration information in the multiple registration options. In response to obtaining the registration information, the control device transmits the registration information to the cloud server via the T-BOX. The cloud server generates the target account and the target account's account data based on the registration information, then establishes a correspondence between the facial features and the target account and stores the target account's account data.

[0110] Of course, the driver can also actively register account data when driving the vehicle for the first time. The registration process is the same as the above registration process and will not be repeated here.

[0111] In an embodiment of the present application, a correspondence between facial features and accounts is pre-established in the cloud server, and the corresponding account is determined based on the facial features. In this way, when the user drives other vehicles, the account data of the corresponding account can also be obtained through the facial features. When the user is driving other vehicles, when the user is fatigued or in other abnormal conditions, the user can also be reminded through the reminder method corresponding to the user, thereby improving the safety of the user during driving.

[0112] Step 203: The control device receives the account data of the target account sent by the cloud server, where the account data includes reminder configuration information corresponding to the abnormal driving status.

[0113] After the cloud server obtains the target account's account data, it sends the target account's account data to the control device through T-BOX. The control device receives and stores the target account's account data locally.

[0114] In an embodiment of the present application, the control device stores the account data of the target account locally, and can subsequently obtain the corresponding reminder method directly from the local account data without the need to interact with the cloud server, which greatly shortens the time.

[0115] In addition, when the cloud server sends the target account's account data to the control device, it also sends the correspondence between the target facial features and the target account. The control device then stores this correspondence locally. This allows the driver to compare facial features directly locally the next time they drive the vehicle, eliminating the need to interact with the cloud server, further reducing the time required. To prevent excessive local storage from affecting device processing speed, the control device can delete less frequently used account data and the correspondence between accounts and facial features at regular intervals.

[0116] Step 204: The control device determines the frequency and duration of the driver's eye closure within a first preset time period based on the facial image.

[0117] In step 201, the control device collects facial images in real time or periodically; thus, the control device acquires multiple facial images. For each facial image, the control device determines the degree of eye closure of the driver in the facial image. Based on the degree of eye closure, the control device determines whether the driver has closed his eyes. If the driver has closed his eyes, the control device determines the frequency and duration of eye closure within a first preset time period based on multiple facial images preceding and following the facial image.

[0118] The control device may determine the eye closure frequency and duration itself or through the DMS, and there is no specific limitation on this. Furthermore, the first preset duration may be set and modified as needed, for example, 30 seconds or 40 seconds, and there is no specific limitation on this.

[0119] Step 205: The control device determines the driver's fatigue information based on the eye-closing frequency and the eye-closing duration. The fatigue information includes a fatigue state and a fatigue level corresponding to the fatigue state.

[0120] In one possible implementation, when the eye-closing frequency reaches a first frequency and the eye-closing duration is greater than a first duration but less than a second duration, the control device determines that the driver's fatigue state is fatigue and the fatigue level is the first level.

[0121] In another possible implementation, when the eye-closing frequency reaches the second frequency and the eye-closing duration is greater than the second duration but less than the third duration, the control device determines that the driver's fatigue state is fatigue and the fatigue level is the second level.

[0122] In another possible implementation, when the eye-closing frequency reaches a third frequency and the eye-closing duration is greater than a third duration, the control device determines that the driver's fatigue state is fatigue and the fatigue level is the third level.

[0123] In the three implementations described above, the third frequency is lower than the second frequency, and the second frequency is lower than the first frequency. Furthermore, the fatigue levels corresponding to the first, second, and third levels increase in order, i.e., the third level of fatigue is greater than the second level of fatigue, and the second level of fatigue is greater than the first level of fatigue. For example, the third level represents severe fatigue, the second level represents moderate fatigue, and the first level represents mild fatigue.

[0124] Among them, the first frequency, the second frequency, the third frequency, the first duration, the second duration and the third duration can all be set and changed as needed. For example, the first frequency is 9 times, the second frequency is 3 times, the third frequency is 2 times, the first duration is 1s, the second duration is 3s, and the third duration is 5s. Accordingly, if the driver closes his eyes 9 times within 30s, and the duration of each eye closure is greater than 1s but less than 2s, the control device determines that the driver is mildly fatigued. If the driver closes his eyes 3 times within 30s, and the duration of each eye closure is greater than 3s but less than 4s, the control device determines that the driver is moderately fatigued. If the driver closes his eyes 2 times within 30s, and the duration of each eye closure is greater than 5s, the control device determines that the driver is severely fatigued.

[0125] In the embodiment of the present application, the control device may first execute steps 202-203 and then execute steps 204-205, or may first execute steps 204-205 and then execute steps 202-203, and there is no specific limitation on this.

[0126] Step 206: When the driver's fatigue state is fatigue, the control device obtains a first correspondence between the fatigue level and the reminder method from the reminder configuration information corresponding to the abnormal driving state.

[0127] The control device locally stores the account data of the target account. When the driver is driving fatigued, the control device obtains a first corresponding relationship from the account data of the locally stored target account. The first corresponding relationship is the corresponding relationship between the fatigue level and the reminder method. Different fatigue levels correspond to different reminder methods.

[0128] If this is the driver's first time driving the vehicle, the reminder method corresponding to the same fatigue level in the first correspondence may be the default reminder method of the control device, and the same fatigue level may correspond to one or more reminder methods. If this is not the driver's first time driving the vehicle, the reminder method corresponding to the same fatigue level in the first correspondence is an effective reminder method, which is a reminder method that can alleviate the driver's fatigue level, and there may be one or more effective reminder methods.

[0129] In the embodiment of the present application, different accounts correspond to different reminder methods in the first correspondence. That is, different drivers are given different reminder methods when they are fatigued. This is because some reminder methods are effective for the first target, while others are effective for the second target. For example, when both the first and second drivers are moderately fatigued, the effective reminder method for the first driver is to control the device to simulate another person and conduct a virtual conversation with the first driver, while the effective reminder method for the second driver is to control the device to control the knocking device to knock on the driver.

[0130] In an embodiment of the present application, the control device establishes a correspondence between fatigue levels and reminder methods according to the effective reminder methods corresponding to the driver, thereby effectively reminding the driver when the driver is driving fatigued, thereby effectively alleviating the driver's fatigue state.

[0131] Step 207: The control device determines a first reminder method corresponding to the fatigue level from the first corresponding relationship based on the fatigue level corresponding to the fatigue state.

[0132] In the case that the driver drives the vehicle for the first time, the control device randomly selects a reminder method from the reminder methods corresponding to the fatigue level in the first correspondence or selects a reminder method in sequence based on the fatigue level.

[0133] For example, the first level of reminders includes sounding an alarm and opening windows for ventilation. The second level includes the control device simulating a virtual conversation with another person, and the control device controlling a knocking device to knock on the driver's back. The third level includes the control device controlling a spraying device to spray liquid in the driver's direction, and the control device taking over the steering wheel to control vehicle driving. The spraying device can be located on the side of the main driver's seat, and the knocking device can be located inside the main driver's seat backrest. When the driver's fatigue level is the first level, the control device randomly selects one of the two reminder methods corresponding to the first level.

[0134] In the case that this is not the first time for the driver to drive the vehicle, the control device determines an effective reminder method corresponding to the fatigue level from the first corresponding relationship based on the fatigue level.

[0135] The first level corresponds to opening windows for ventilation, the second level corresponds to controlling the device to simulate a virtual conversation with another person, and the third level corresponds to controlling the spray device to spray liquid in the driver's direction. If the driver's fatigue level is level two, the control device selects the effective reminder method corresponding to level two.

[0136] The above are only a few reminder methods. In actual applications, other reminder methods can also be used to remind the driver, for example, playing music or increasing the music volume, etc., which are not specifically limited.

[0137] Step 208: The control device reminds the driver based on the first reminder method.

[0138] In an embodiment of the present application, when the driver is driving the vehicle for the first time, the control device reminds the driver based on the first reminder method corresponding to the fatigue level, and then determines whether the driver's fatigue level is relieved within the second preset time period, that is, whether the fatigue level has decreased. In the case where the driver's fatigue level has decreased, for example, from moderate fatigue to mild fatigue, or from mild fatigue to wakefulness, the control device uses the reminder method as the effective reminder method and updates the first corresponding relationship based on the effective reminder method. In the case where the driver's fatigue level has not decreased, the control device reselects the first reminder method from the reminder methods corresponding to the fatigue level, and reminds the driver based on the reselected first reminder method until a valid reminder method is selected.

[0139] In the case that this is not the first time that the driver is driving the vehicle, the control device will remind the driver based on the effective reminder method, and then determine whether the driver's fatigue level has been alleviated within the second preset time period, that is, whether the fatigue level has decreased. If the driver's fatigue level has decreased, no other operations are required. If the driver's fatigue level has not decreased, and there are multiple effective reminder methods corresponding to the fatigue level, the control device will reselect the effective reminder method and remind the driver based on the reselected effective reminder method until an effective reminder method that can reduce the driver's fatigue level is selected. If the driver's fatigue level has not decreased, and there is only one effective reminder method corresponding to the fatigue level, the control device will select the first reminder method from the default reminder method to remind the driver.

[0140] The process of the control device determining whether the driver's fatigue level is relieved within the second preset time period is the same as the process of determining the driver's fatigue state, which will not be repeated here.

[0141] In an embodiment of the present application, when the control device takes over the steering wheel, the control device controls the vehicle's driving to remind the driver. For example, the control device may first output a voice message to remind the driver that the control device is about to take over the steering wheel, and then the control device controls the vehicle's driving.

[0142] The control device can control the vehicle's travel based on the type of road it is on. For example, if the vehicle is on a highway, the control device can determine the location of the service area closest to the current location and control the vehicle to travel to the service area. For another example, if the vehicle is on an urban road, the control device can determine the nearest parking lot or permitted parking area and control the vehicle to travel to the parking lot or permitted parking area.

[0143] The present application provides a driving status monitoring method. When a driver is driving a vehicle, the method uses facial recognition technology to obtain the driver's target account data from a cloud server. This account data includes reminder configuration information corresponding to abnormal driving conditions, thereby obtaining a reminder method suitable for the driver's abnormal driving conditions. Furthermore, when facial recognition technology identifies driver fatigue, the method uses the reminder configuration information corresponding to the driver's abnormal condition to obtain a reminder method suitable for the user in the fatigue state, thereby reminding the driver. This can improve the effectiveness of the reminder, thereby ensuring safe driving for the user and reducing the incidence of traffic accidents.

[0144] In this embodiment, the user registers an account through facial recognition when using the vehicle for the first time. Subsequent logins allow for seamless login and access to the pre-set account data. During daily driving, the system automatically identifies the user's age, gender, and driving behavior, and uses scene-based brain technology to make decisions, providing reminders through voice broadcasts, phone calls, music, and other means. These proactive reminders enhance driving safety and provide effective driving pleasure.

[0145] Next, let me introduce the process of monitoring distraction.

[0146] In an embodiment of the present application, the control device can determine the actual gaze area of ​​the driver's eyes based on the facial image; when the actual gaze area does not match the theoretical gaze area, determine that the driver is in a distracted state; obtain a second correspondence between the abnormal state and the reminder method from the reminder configuration information corresponding to the abnormal driving state; obtain a second reminder method corresponding to the distracted state from the second correspondence; and remind the driver based on the second reminder method.

[0147] In this implementation, the control device extracts eye features from the facial image, determines the driver's line of sight direction based on the eye features, and determines the actual gaze area based on the line of sight direction; then determines whether the actual gaze area is the theoretical gaze area. If the actual gaze area is the theoretical gaze area, it is determined that the driver is not in a distracted state. If the actual gaze area is not the theoretical gaze area, that is, the actual gaze area does not match the theoretical gaze area, it is determined that the driver is in a distracted state.

[0148] The theoretical gaze area can be set and changed as needed. For example, the theoretical gaze area is a rearview mirror, a front windshield, a reflector, an instrument panel, etc., and there is no specific limitation on this.

[0149] When the control device cannot determine the driver's actual line of sight direction based on the eye features, the control device can determine the actual gaze area based on the driver's head posture direction, and then determine whether the driver is in a distracted state based on the actual gaze area.

[0150] If the driver is distracted, the control device retrieves a second correspondence from the reminder configuration information corresponding to the abnormal driving state. The second correspondence is a correspondence between abnormal states and reminder methods, with one abnormal state corresponding to one or more reminder methods. The control device retrieves a second reminder method corresponding to the distracted state from the second correspondence and reminds the driver based on the second reminder method.

[0151] If there is only one second reminder method corresponding to the distracted state, the control device will directly remind the driver based on the second reminder method. If there are multiple second reminder methods corresponding to the distracted state, the control device may randomly select one reminder method from the multiple second reminder methods to remind the driver. After reminding the driver using the selected second reminder method, it is determined whether the driver is still in a distracted state. If the driver is still in a distracted state, a second reminder method is reselected from the multiple second reminder methods until the driver changes from a distracted state to a non-distracted state after being reminded based on the second reminder method. If the driver is no longer distracted, the second reminder method is marked and will be used as a priority to remind the driver in the future. The second reminder method can be set and changed as needed, and there is no specific limitation on this.

[0152] The following describes the call status monitoring process.

[0153] In an embodiment of the present application, the control device detects the driver's hand based on the face image; when it is detected that the driver's hand is holding the electronic device, the control device determines the distance between the driver's hand and the ear; when the distance is less than a preset distance, it is determined that the driver is in a call state; a third reminder method corresponding to the call state is obtained from the second corresponding relationship; the driver is reminded based on the third reminder method; when the distance is not less than the preset distance, it is determined that the driver is not in a call state.

[0154] In this implementation, the control device detects the driver's hand from the facial image. If the driver's hand is detected and the hand is holding an object, it determines whether the object is an electronic device. If the object is an electronic device, it determines that the driver's hand is holding an electronic device. In this case, the control device determines the distance between the driver's hand and ear. If the distance is less than a preset distance, it determines that the driver is in a call state. Alternatively, the control device can input the facial image into a recognition model and use the recognition model to score the driver's call behavior. If the score exceeds a first scoring threshold, it determines that the driver is in a call state; if the score does not exceed the first scoring threshold, it determines that the driver is not in a call state.

[0155] Next, we will introduce the process of monitoring smoking status.

[0156] In an embodiment of the present application, the control device detects the driver's hands based on the facial image; if it is detected that the driver's hands are holding a smoking object, the driver's mouth is detected; if it is detected that the smoking object is in the driver's mouth, it is determined that the driver is in a smoking state; a fourth reminder method corresponding to the smoking state is obtained from the second correspondence; and the driver is reminded based on the fourth reminder method.

[0157] In addition, the control device can also obtain the smoke concentration in the vehicle, and determine the vehicle's environment when the smoke concentration is greater than a preset concentration; determine the ventilation method based on the vehicle's environment; and ventilate the vehicle based on the ventilation method.

[0158] In this implementation, the control device detects the driver's hand status from the facial image. If the driver's hand is detected and the hand is holding an object, it determines whether the object is a smoking object. If the object is a smoking object, the driver's mouth is detected. If the smoking object is detected in the driver's mouth, the driver is determined to be smoking. Alternatively, the control device may input the facial image into a recognition model, which uses the recognition model to score the driver's smoking behavior. If the score exceeds a second scoring threshold, the driver is determined to be smoking; if the score does not exceed the second scoring threshold, the driver is determined to be not smoking.

[0159] It should be noted that the process of the control device reminding the driver based on the third reminder method or the fourth reminder method is the same as the process of reminding the driver based on the second reminder method, and will not be repeated here.

[0160] Furthermore, if the driver is smoking, the control device can also detect the smoke concentration through sensors within the vehicle. If the smoke concentration exceeds a preset concentration, the control device determines the vehicle's environment. Different environments correspond to different ventilation methods. Ventilation is performed based on the corresponding ventilation method, thereby improving the air quality inside the vehicle. For example, if the vehicle is in a rainy or snowy environment, the ventilation method can be to turn on the air conditioner. If the vehicle is in a sunny environment, the ventilation method can be to open the windows for ventilation.

[0161] In this embodiment of the present application, the target account's account data also includes the driver's identity information; accordingly, the control device can also broadcast or recommend music to the driver based on the driver's emotion type and identity information. This process can include: the control device determines the driver's emotion type based on facial features; if the driver's emotion type is positive, the control device obtains the driver's identity information from the target account's account data; and based on the positive emotion and identity information, broadcasts a voice message or recommends music to the driver.

[0162] In this implementation, the control device determines the driver's mouth features based on facial features. If the driver's mouth features are upturned corners of the mouth, slightly open mouth, or open mouth with teeth exposed, it is determined that the driver is in a happy state, that is, the driver's emotion type is positive emotion.

[0163] When the driver's emotion type is positive, the control device obtains the driver's identity information, which includes gender, age, etc. Based on the positive emotion and identity information, the control device recommends music that matches the positive emotion and identity information to the driver, or broadcasts a voice message that matches the positive emotion and identity information.

[0164] In this embodiment of the present application, the target account's account data also includes vehicle configuration data; accordingly, the control device can also automatically adjust the position of the seat and rearview mirror. This process can include: the control device obtains the position information of the main driver's seat and the position information of the rearview mirror from the vehicle configuration data; and adjusts the position of the main driver's seat and the rearview mirror based on the position information of the main driver's seat and the rearview mirror.

[0165] In this implementation, if this is not the driver's first time driving the vehicle, the vehicle configuration data stores the driver's historical position information for the driver's seat and rearview mirrors. After the driver enters the vehicle, the control device automatically adjusts the position of the driver's seat and rearview mirrors based on this historically stored position information.

[0166] When the driver drives the vehicle for the first time, the driver can manually adjust the position of the main driver's seat and the position of the rearview mirror, and the control device stores the position of the main driver's seat and the position of the rearview mirror. In this way, the position of the main driver's seat and the position of the rearview mirror can be automatically adjusted the next time the driver drives the vehicle to meet the driver's needs.

[0167] In an embodiment of the present application, the control device can also obtain the height of the steering wheel from the account data of the target account, and automatically adjust the height of the steering wheel based on the obtained steering wheel height after the driver gets in the car.

[0168] In an embodiment of the present application, the control device can also heat the driver's seat. This process can be as follows: when the vehicle is powered on, the control device obtains the vehicle's external temperature. If the temperature is less than a preset temperature, the driver's seat heating function is activated; if the temperature is not less than the preset temperature, the driver's seat heating function is deactivated. Alternatively, when the vehicle is powered on, the control device displays an interactive interface with a function button for turning on heating. In response to detecting that the function button for turning on heating has been activated, the control device activates the driver's seat heating function.

[0169] Among them, after the control device turns on the heating function of the main driver's seat, it can output a voice message and / or display a message pop-up box on the interactive interface, so that the user knows that the heating function has been turned on.

[0170] The main driver's seat can also support more seat posture adjustments. In addition to the regular adjustments of level, height and backrest, it also supports adjustments in the direction of leg rest, shoulders, etc. to achieve a comfortable sitting posture. It also supports heating, ventilation, massage, memory and other functions.

[0171] In the embodiment of the present application, algorithms such as fatigue perception, gaze perception, distraction perception, and behavior perception can be used to monitor the driver's driving status, and the driver can be reminded in time when the driver is in an abnormal driving state, thereby ensuring the driver's safe driving.

[0172] Furthermore, core technologies such as image perception, voice signal processing, speech recognition, and scene-based brains can be leveraged to deliver next-generation in-vehicle AI services, including predictions, proactive recommendations, voice dialogue, and behavioral prediction. For example, if the driver is happy, the control device can proactively provide driver care. If the driver is smoking, active ventilation can be activated to improve in-car air quality. Furthermore, frequently used settings can be automatically adjusted based on the driver's driving habits, enhancing the user experience.

[0173] FIG5 is a schematic diagram of the structure of a driving state monitoring device provided in an embodiment of the present application. Referring to FIG5 , the device includes:

[0174] A first acquisition module 501 is used to acquire a facial image of a vehicle driver;

[0175] Extraction module 502 is configured to extract facial features of the driver based on the facial image and send the facial features to a cloud server. The cloud server is configured to determine a target account of the driver based on the facial features and obtain account data of the target account, including reminder configuration information corresponding to abnormal driving conditions.

[0176] Receiving module 503, used to receive the account data of the target account sent by the cloud server;

[0177] A first determining module 504 is configured to determine, based on the facial image, the frequency and duration of eye closure of the driver within a first preset time period;

[0178] A second determining module 505 is configured to determine the driver's fatigue information based on the eye closing frequency and the eye closing duration, where the fatigue information includes a fatigue state and a fatigue level corresponding to the fatigue state;

[0179] The second acquisition module 506 is used to acquire a first correspondence between a fatigue level and a reminder method from the account data of the target account when the driver is driving fatigued;

[0180] A third determining module 507 is configured to determine, based on the fatigue level corresponding to the fatigue state, a first reminder method corresponding to the fatigue level from the first corresponding relationship;

[0181] The reminder module 508 is configured to remind the driver based on a first reminder method.

[0182] In one possible implementation, the second determination module 505 is used to determine that the driver's fatigue state is fatigue and the fatigue level is the first level when the eye-closing frequency reaches a first frequency and the eye-closing duration is greater than the first duration but less than the second duration; to determine that the driver's fatigue state is fatigue and the fatigue level is the second level when the eye-closing frequency reaches a second frequency and the eye-closing duration is greater than the second duration but less than a third duration; to determine that the driver's fatigue state is fatigue and the fatigue level is the third level when the eye-closing frequency reaches a third frequency and the eye-closing duration is greater than the third duration; wherein the third frequency is less than the second frequency, the second frequency is less than the first frequency, and the fatigue degrees corresponding to the first level, the second level and the third level increase respectively.

[0183] In another possible implementation, the apparatus further includes:

[0184] A fourth determination module is used to determine the actual gaze area of ​​the driver based on the facial image;

[0185] a fifth determination module, configured to determine that the driver is in a distracted state if the actual gaze area does not match the theoretical gaze area;

[0186] A third acquisition module is used to obtain a second correspondence between the abnormal state and the reminder method from the reminder configuration information corresponding to the abnormal driving state;

[0187] A fourth acquisition module, configured to acquire a second reminder method corresponding to the distracted state from the second corresponding relationship;

[0188] The reminding module 508 is further configured to remind the driver based on the second reminding method.

[0189] In another possible implementation, the apparatus further includes:

[0190] A first detection module is used to detect the driver's hands based on the face image;

[0191] a sixth determination module, configured to determine the distance between the driver's hand and the ear when detecting that the driver's hand is holding the electronic device;

[0192] a seventh determination module, configured to determine that the driver is in a call state when the distance is less than a preset distance;

[0193] A fifth obtaining module, configured to obtain a third reminder method corresponding to the call status from the second corresponding relationship;

[0194] The reminding module 508 is further configured to remind the driver based on a third reminding method.

[0195] In another possible implementation, the apparatus further includes:

[0196] The second detection module is used to detect the driver's mouth when it is detected that the driver's hand is holding a smoking object;

[0197] an eighth determining module, configured to determine that the driver is smoking when a smoking object is detected in the driver's mouth;

[0198] an eighth determination module, configured to obtain a smoke concentration in the vehicle and, if the smoke concentration is greater than a preset concentration, determine the vehicle's environment;

[0199] a ninth determination module, configured to determine a ventilation mode based on the environment in which the vehicle is located;

[0200] The ventilation module is used to ventilate the vehicle based on a ventilation mode.

[0201] In another possible implementation, the account data of the target account also includes the driver's identity information; and the device further includes:

[0202] a tenth determination module, configured to determine the driver's emotion type based on facial features;

[0203] a sixth acquisition module, configured to acquire the driver's identity information from the account data of the target account when the driver's emotion type is positive;

[0204] The recommendation module is used to broadcast voice messages or recommend music to the driver based on positive emotions and identity information.

[0205] In another possible implementation, the account data of the target account also includes vehicle configuration data; and the device further includes:

[0206] a seventh acquisition module, configured to acquire position information of a driver's seat and a rearview mirror from vehicle body configuration data;

[0207] The adjustment module is used to adjust the position of the main driver's seat and the position of the rearview mirror based on the position information of the main driver's seat and the position information of the rearview mirror.

[0208] An embodiment of the present application provides a driving status monitoring device. When a driver is driving a vehicle, the device uses facial recognition technology to obtain the driver's target account data from a cloud server. The account data includes reminder configuration information corresponding to abnormal driving conditions, thereby obtaining a reminder method suitable for the driver's abnormal driving conditions. When the facial recognition technology identifies the driver as fatigued, the device uses the reminder configuration information corresponding to the abnormal condition to obtain a reminder method suitable for the user in the fatigue state, thereby reminding the driver. This can improve the effectiveness of the reminder, thereby ensuring the user's safe driving and reducing the incidence of traffic accidents.

[0209] It should be noted that the driving state monitoring device provided in the above embodiment is merely illustrated by the division of the aforementioned functional modules when performing driving state monitoring. In actual applications, the aforementioned functions can be assigned to different functional modules as needed, that is, the internal structure of the control device can be divided into different functional modules to perform all or part of the functions described above. Furthermore, the driving state monitoring device provided in the above embodiment and the driving state monitoring method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0210] The structural block diagram of the control device can be seen in Figure 6. The control device 600 may vary significantly due to different configurations or performance, and may include a processor (central processing unit, CPU) 601 and a memory 602. The memory 602 stores at least one program code, which is loaded and executed by the processor 601 to implement the driving state monitoring method in the above embodiment. Of course, the control device 600 may also have components such as a wired or wireless network interface, a keyboard, and input and output interfaces for input and output. The control device 600 may also include other components for implementing device functions, which are not detailed here.

[0211] In an exemplary embodiment, a computer-readable storage medium is further provided, the computer-readable storage medium storing at least one program code, which is loaded and executed by a processor to implement the driving state monitoring method of the above embodiment. Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, an optical data storage device, or the like.

[0212] In an exemplary embodiment, a computer program product is further provided. The computer program product stores at least one program code, and the at least one program code is loaded and executed by a processor to implement the driving state monitoring method in the above embodiment.

[0213] In an exemplary embodiment, the computer program product involved in the embodiments of the present application can be deployed and executed on a control device, or on multiple control devices located at one location, or on multiple control devices distributed at multiple locations and interconnected through a communication network. Multiple control devices distributed at multiple locations and interconnected through a communication network can constitute a blockchain system.

[0214] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0215] The above description is only for the purpose of facilitating those skilled in the art to understand the technical solution of this application and is not intended to limit this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application shall be included in the scope of protection of this application.

Claims

1. A driving state monitoring method, characterized in that: The method comprises: Acquire a facial image of the driver of the vehicle; Based on the facial image, extract the facial features of the driver, and send the facial features to a cloud server; wherein the cloud server is used to determine a target account of the driver based on the facial features, and obtain account data of the target account, wherein the account data includes reminder configuration information corresponding to the abnormal driving state; Receiving the account data of the target account sent by the cloud server; Based on the facial image, determining the frequency and duration of eye closure of the driver within a first preset time period; Based on the eye closing frequency and the eye closing duration, determining fatigue information of the driver, the fatigue information including a fatigue state and a fatigue level corresponding to the fatigue state; In the case where the fatigue state is fatigue, obtaining a first correspondence between a fatigue level and a reminder method from reminder configuration information corresponding to the abnormal driving state; Based on the fatigue level corresponding to the fatigue state, determining a first reminder method corresponding to the fatigue level from the first corresponding relationship; The driver is reminded based on the first reminder method.

2. The method according to claim 1, characterized in that The determining of the driver's fatigue information based on the eye closing frequency and the eye closing duration includes: When the eye-closing frequency reaches the first frequency and the eye-closing duration is greater than the first duration but less than the second duration, determining that the driver's fatigue state is fatigue and the fatigue level is the first level; When the eye-closing frequency reaches a second frequency and the eye-closing duration is greater than the second duration but less than a third duration, determining that the driver's fatigue state is fatigue and the fatigue level is a second level; When the eyes closing frequency reaches the third frequency and the eyes closing duration is greater than the third duration, it is determined that the driver's fatigue state is fatigue and the fatigue level is the third level; wherein the third frequency is less than the second frequency, the second frequency is less than the first frequency, and the fatigue degrees corresponding to the first level, the second level and the third level increase respectively.

3. The method according to claim 1, characterized in that The method further comprises: Based on the facial image, determining the actual gaze area of ​​the driver's line of sight; In the case where the actual gaze area does not match the theoretical gaze area, determining that the driver is in a distracted state; Acquire a second correspondence between the abnormal state and the reminder method from the reminder configuration information corresponding to the abnormal driving state; Acquire a second reminder method corresponding to the distraction state from the second corresponding relationship; The driver is reminded based on the second reminder method.

4. The method according to claim 3, characterized in that: The method further comprises: Based on the facial image, detecting the driver's hands; In the case where it is detected that the driver's hand is holding the electronic device, determining the distance between the driver's hand and the ear; When the distance is less than a preset distance, determining that the driver is in a talking state; Acquire a third reminder method corresponding to the call status from the second corresponding relationship; The driver is reminded based on the third reminder method.

5. The method according to claim 4, characterized in that The method further comprises: In the case where it is detected that the driver's hand is holding a smoking object, detecting the driver's mouth; In the case where the smoking object is detected to be contained in the driver's mouth, determining that the driver is in a smoking state; Acquiring the smoke concentration in the vehicle, and determining the environment in which the vehicle is located when the smoke concentration is greater than a preset concentration; Determining a ventilation method based on the environment in which the vehicle is located; The vehicle is ventilated based on the ventilation method.

6. The method according to claim 1, characterized in that The account data of the target account also includes the identity information of the driver; the method further includes: Determining the driver's emotion type based on the facial features; When the driver's emotion type is positive emotion, obtaining the driver's identity information from the account data of the target account; Based on the positive emotion and the identity information, a voice message is broadcasted or music is recommended to the driver.

7. The method according to claim 1, characterized in that The account data of the target account also includes vehicle body configuration data; the method further includes: Acquire the position information of the main driver's seat and the position information of the rearview mirror from the vehicle body configuration data; Based on the position information of the main driver's seat and the position information of the rearview mirror, the position of the main driver's seat and the position of the rearview mirror are adjusted.

8. A driving status monitoring device, characterized in that: The device comprises: A first acquisition module, used to acquire a facial image of a driver of the vehicle; an extraction module, configured to extract facial features of the driver based on the facial image, and send the facial features to a cloud server; wherein the cloud server is configured to determine a target account of the driver based on the facial features, and obtain account data of the target account, wherein the account data includes reminder configuration information corresponding to an abnormal driving state; A receiving module, used for receiving the account data of the target account sent by the cloud server; A first determination module, configured to determine, based on the facial image, the frequency and duration of eye closure of the driver within a first preset duration; A second determination module, configured to determine the driver's fatigue state based on the eye closing frequency and the eye closing duration; A second acquisition module is used to acquire a first correspondence between a fatigue level and a reminder method from the reminder configuration information corresponding to the abnormal driving state when the driver is driving fatigued; a third determining module, configured to determine, based on the fatigue level corresponding to the fatigue state, a reminder method corresponding to the fatigue level from the first corresponding relationship; A reminder module is used to remind the driver based on the reminder method.

9. The device according to claim 8, characterized in that The second determination module is used to determine that the driver's fatigue state is fatigue and the fatigue level is the first level when the eyes closing frequency reaches the first frequency and the eyes closing duration is greater than the first duration but less than the second duration; determine that the driver's fatigue state is fatigue and the fatigue level is the second level when the eyes closing frequency reaches the second frequency and the eyes closing duration is greater than the second duration but less than the third duration; determine that the driver's fatigue state is fatigue and the fatigue level is the third level when the eyes closing frequency reaches the third frequency and the eyes closing duration is greater than the third duration; wherein the third frequency is less than the second frequency, the second frequency is less than the first frequency, and the fatigue degrees corresponding to the first level, the second level and the third level respectively increase in sequence.

10. The device according to claim 8, characterized in that The device also includes: A fourth determination module, configured to determine an actual gaze area of ​​the driver's sight based on the face image; A fifth determination module, configured to determine that the driver is in a distracted state when the actual gaze area does not match the theoretical gaze area; A third acquisition module, used to acquire a second corresponding relationship between the abnormal state and the reminder method from the reminder configuration information corresponding to the abnormal driving state; a fourth acquisition module, configured to acquire a second reminder method corresponding to the distraction state from the second corresponding relationship; The reminding module is further used to remind the driver based on the second reminding method.

11. The device according to claim 10, characterized in that The device also includes: A first detection module, configured to detect the driver's hands based on the face image; a sixth determination module, configured to determine the distance between the driver's hand and the ear when it is detected that the driver's hand is holding the electronic device; A seventh determination module, configured to determine that the driver is in a talking state when the distance is less than a preset distance; A fifth acquisition module, configured to acquire a third reminder method corresponding to the call status from the second corresponding relationship; The reminding module is further used to remind the driver based on the third reminding method.

12. The device according to claim 11, characterized in that The device also includes: A second detection module is used to detect the driver's mouth when it is detected that the driver's hand is holding a smoking object; an eighth determination module, configured to determine that the driver is in a smoking state when it is detected that the smoking object is contained in the driver's mouth; an eighth determination module, configured to obtain the smoke concentration in the vehicle, and determine the environment in which the vehicle is located when the smoke concentration is greater than a preset concentration; A ninth determination module, configured to determine a ventilation mode based on the environment in which the vehicle is located; A ventilation module is used to ventilate the vehicle based on the ventilation method.

13. The device according to claim 8, characterized in that The account data of the target account also includes the identity information of the driver; the device also includes: a tenth determination module, configured to determine the driver's emotion type based on the facial features; a sixth acquisition module, configured to acquire the identity information of the driver from the account data of the target account when the emotion type of the driver is a positive emotion; A recommendation module is used to broadcast a voice message or recommend music to the driver based on the positive emotion and the identity information.

14. The device according to claim 8, characterized in that The account data of the target account also includes vehicle body configuration data; the device also includes: a seventh acquisition module, used to acquire the position information of the main driver's seat and the position information of the rearview mirror from the vehicle body configuration data; An adjustment module is used to adjust the position of the main driver's seat and the position of the rearview mirror based on the position information of the main driver's seat and the position information of the rearview mirror.

15. A control device, characterized in that: The control device includes a processor and a memory, wherein at least one program code is stored in the memory, and the at least one program code is loaded and executed by the processor to implement the driving state monitoring method as described in any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that: At least one program code is stored in the computer-readable storage medium, and the at least one program code is loaded and executed by the processor to implement the driving state monitoring method as described in any one of claims 1 to 7.

17. A computer program product, characterized in that The computer program product stores at least one program code, and the at least one program code is loaded and executed by a processor to implement the driving state monitoring method as described in any one of claims 1 to 7.

Images