Driver state detection method, vehicle control method, and vehicle

Abstract

A driver state detection method, a vehicle control method and a vehicle are disclosed, and relate to the technical field of driver monitoring, and include: acquiring multiple frames of face images of a driver within a first time length, and determining a driver state data set according to the multiple frames of face images; determining at least one of a number of squeezed eye blinks, a number of complete eye blinks, a number of incomplete eye blinks and a number of eye rubs of the driver according to the driver state data set; and determining an eye state of the driver according to the at least one of the number of squeezed eye blinks, the number of complete eye blinks, the number of incomplete eye blinks and the number of eye rubs. The application can detect whether the driver is continuously in an eye discomfort state in time, and improve the accuracy of determining the eye state of the driver, so as to avoid accidents.

Description

Technical Field

[0001] This application relates to the field of driver monitoring technology, and more specifically, to a driver state detection method, a vehicle control method, and a vehicle. Background Technology

[0002] In recent years, with the rapid development of technology, the use of autonomous driving and advanced driver assistance systems (ADAS) in vehicles has become increasingly widespread, and non-contact driver monitoring systems (DMS) based on in-vehicle cameras have gradually become a research focus. DMS primarily uses near-infrared cameras to continuously capture facial images, and then extracts biometric features such as eyelid opening and closing, blinking frequency, gaze direction, and pupillary light reflex based on deep learning models. This allows for real-time warnings of whether the driver is fatigued, distracted, or driving under the influence of alcohol or drugs. Currently, because drivers are under intense visual strain while driving, they are significantly more likely to experience eye discomfort than other groups. Failure to detect persistent eye discomfort in a timely manner could lead to serious accidents. Summary of the Invention

[0003] In view of this, embodiments of this application propose a driver state detection method and a vehicle control method and vehicle to improve the above-mentioned problems.

[0004] According to a first aspect of the embodiments of this application, a driver state detection method is provided, the method comprising: acquiring multiple frames of facial images of the driver within a first time period, and determining a driver state dataset based on the multiple frames of facial images; determining at least one of the driver's squeeze blink count, full blink count, incomplete blink count, and eye rubbing count based on the driver state dataset; and determining the driver's eye state based on at least one of the squeeze blink count, full blink count, incomplete blink count, and eye rubbing count.

[0005] In some embodiments, the method further includes: if the driver's eye condition is eye discomfort, adjusting the vehicle's interior environment information; the interior environment information includes interior humidity, or the interior environment information includes interior humidity and one or more of the following: interior temperature, air conditioning direction, air conditioning speed, interior display parameters, or interior lighting parameters.

[0006] In some embodiments, adjusting the vehicle's interior environment information if the driver's eye condition is eye discomfort includes: controlling the vehicle's humidity control system to adjust the interior humidity when the driver is in an eye discomfort state and the driver has dry eye symptoms, wherein the humidity control system is used to adjust the humidification area inside the vehicle to match the position of the driver's eyes in the cabin.

[0007] In some embodiments, the driver state dataset includes at least one or more combinations of the following: eye aspect ratio, mouth aspect ratio, number of eye rubbing parameters, blinking frequency, or distance between the hand and the eye.

[0008] In some embodiments, where the driver state dataset includes eye aspect ratio, determining at least one of the driver's squeeze blink count, full blink count, incomplete blink count, and eye rubbing count based on the driver state dataset includes: determining the squeeze blink count based on the multi-frame facial images and the eye aspect ratio of each frame of the facial images; and / or, obtaining a reference value for the eye aspect ratio and determining a closed-eye threshold and an incomplete blink threshold based on the reference value; wherein the incomplete blink threshold is higher than the closed-eye threshold; comparing the eye aspect ratio of each frame of the facial images with the closed-eye threshold to determine the blink count; and / or, comparing the eye aspect ratio of each frame of the facial images, the closed-eye threshold, and the incomplete blink threshold to determine the incomplete blink count.

[0009] In some embodiments, the squeezing blink is determined based on preset conditions corresponding to at least one of the following parameters: total blink duration, minimum eye aspect ratio, cumulative tremor intensity of changes in eye aspect ratio, degree of contraction in the periorbital region, and mouth aspect ratio; before determining the number of squeezing blinks, the method further includes: performing a squeezing blink recognition operation on the facial image; when the features of the facial image meet a first condition, determining that the facial image contains a squeezing blink action.

[0010] In some embodiments, the first condition includes: the target duration of blinking is within the duration range, the minimum eye aspect ratio is the eye aspect ratio threshold, the target degree of contraction of the periorbital region is within the target degree of contraction range, the change amplitude of the cumulative tremor intensity is within the target amplitude range, and the facial state indicated by the mouth aspect ratio is that the driver has not performed the target action.

[0011] In some embodiments, determining the driver's eye state based on at least one of the number of squeeze blinks, the number of complete blinks, the number of incomplete blinks, and the number of eye rubbing includes: determining a score corresponding to each number, and determining the driver's eye detection score based on the score corresponding to each number; if the eye detection score reaches a score threshold, then determining that the eye state indicates that the driver is in an eye discomfort state.

[0012] According to a second aspect of the embodiments of this application, a vehicle control method is provided, the method comprising: acquiring historical travel data of the vehicle and historical eye data of the driver, and determining the total driving time of the driver in a first time period based on the historical travel data; determining the probability of the driver experiencing eye discomfort based on the total driving time and the historical eye data; the probability being determined by at least one of the number of squeeze blinks, the number of complete blinks, the number of incomplete blinks, and the number of eye rubbings indicated by the historical eye data; determining the driver's eye detection strategy based on the probability, and executing travel planning for the vehicle based on the eye detection strategy; the travel planning including vehicle control behavior when the vehicle is in a stopable area or a non-stopable area.

[0013] In some embodiments, determining the probability of the driver experiencing eye discomfort based on the total driving time and the historical eye data includes: determining the frequency of the driver experiencing eye discomfort within a first time period based on the historical eye data; determining the probability based on the frequency if the total driving time is greater than or equal to the first time period; and determining the probability cumulatively based on the total driving time and the frequency if the total driving time is greater than or equal to the second time period and less than the first time period, wherein the second time period is less than the first time period.

[0014] In some embodiments, determining the driver's eye detection strategy based on the probability includes: determining the risk level of the driver experiencing eye discomfort based on the probability; if the risk level is a first level or a second level, performing eye detection on the driver and determining the driver's eye detection strategy; wherein the probability of the second level indication is less than the probability of the first level indication; if the risk level is a third level, not performing eye detection on the driver, wherein the probability of the third level indication is less than the probability of the second level indication.

[0015] In some embodiments, the step of performing trip planning for the vehicle according to the eye detection strategy includes: obtaining the vehicle's location information; if the location information indicates that the vehicle is in a non-parking area, determining the travel time of the vehicle from the non-parking area to a parking area, and providing an intervention prompt to the driver based on the travel time, wherein the intervention prompt is used to remind the driver to take preventive measures against eye discomfort; if the location information indicates that the vehicle is in a parking area, providing an immediate prompt to the driver.

[0016] According to a third aspect of the embodiments of this application, a driver state detection device is provided. The device includes: a first acquisition module, configured to acquire multiple frames of facial images of the driver within a first time period, and determine a driver state dataset based on the multiple frames of facial images; a first determination module, configured to determine at least one of the driver's squeeze blink count, full blink count, incomplete blink count, and eye rubbing count based on the driver state dataset; and a second determination module, configured to determine the driver's eye state based on at least one of the squeeze blink count, full blink count, incomplete blink count, and eye rubbing count.

[0017] According to a fourth aspect of the embodiments of this application, a vehicle control device is provided, the device comprising: a second acquisition module, configured to acquire historical travel data of the vehicle and historical eye data of the driver, and determine the total driving time of the driver in a first time period based on the historical travel data; a third determination module, configured to determine the probability of the driver experiencing eye discomfort based on the total driving time and the historical eye data; the probability is determined by at least one of the number of squeeze blinks, the number of complete blinks, the number of incomplete blinks, and the number of eye rubbings indicated by the historical eye data; and a control module, configured to determine the driver's eye detection strategy based on the probability, and execute travel planning for the vehicle based on the eye detection strategy; the travel planning includes vehicle control behavior when the vehicle is in a stopable area or a non-stopable area.

[0018] According to a fifth aspect of the present application, a vehicle is provided, comprising: a processor; and a memory storing computer-readable instructions, wherein when executed by the processor, the computer-readable instructions implement the driver state detection method and / or the vehicle control method as described above.

[0019] According to a sixth aspect of the present application, a computer-readable storage medium is provided that stores computer-readable instructions thereon, which, when executed by a processor, implement the driver state detection method and / or the vehicle control method as described above.

[0020] In the solution of this application, in the embodiments of this application, the driver's state dataset is determined by analyzing multiple frames of facial images of the driver within a first time period. Based on the state dataset, parameters indicating the driver's eye state can be determined, thereby improving the accuracy of determining the driver's eye state and enabling timely detection of whether the driver is continuously experiencing eye discomfort, thus preventing accidents from occurring.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit the embodiments of this application. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0023] Figure 1 This application illustrates a vehicle control system according to an embodiment of the present application; Figure 2 A flowchart illustrating a driver state detection method according to an embodiment of this application is shown; Figure 3 A flowchart illustrating a driver state detection method according to an embodiment of this application is shown; Figure 4 A flowchart illustrating a driver state detection method according to another embodiment of this application is shown; Figure 5 A block diagram of a driver status detection device according to an embodiment of this application is shown; Figure 6 A schematic flowchart of a vehicle control method according to an embodiment of this application is shown; Figure 7 A schematic flowchart of a vehicle control method according to another embodiment of this application is shown; Figure 8 A schematic flowchart of a vehicle control method according to another embodiment of this application is shown; Figure 9 A schematic flowchart of a vehicle control method provided in another embodiment of this application is shown; Figure 10 A block diagram of a vehicle control device according to an embodiment of this application is shown; Figure 11 This is a schematic flowchart of a vehicle control method according to another embodiment of this application; Figure 12 This is a schematic flowchart of a method for detecting eye discomfort according to another embodiment of this application; Figure 13 This is a hardware structure diagram of a vehicle provided according to an embodiment of this application.

[0024] The accompanying drawings have illustrated specific embodiments of the present application. More detailed descriptions will follow. These drawings and descriptions are not intended to limit the scope of the present application's embodiments in any way, but rather to illustrate the concepts of the present application's embodiments to those skilled in the art through specific embodiments. Detailed Implementation

[0025] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0026] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0027] Please see Figure 1 , Figure 1 This application illustrates a vehicle control system according to an embodiment of the present application, such as... Figure 1 As shown below, the vehicle control system implements a vehicle control method by way of example.

[0028] In one alternative implementation, the vehicle control system 100 includes an image acquisition module 110, a processing module 120, and a control module 130, wherein the processing module 120 and the control module 130 refer to software units or modules, and the image acquisition module 110 refers to a hardware device.

[0029] For example, the image acquisition module 110 acquires facial image data of the driver within a first time period, and then sends the facial image data to the processing module 120. The processing module 120 then determines a driver state dataset based on the facial image data, including at least one of the following: the number of squeezed blinks, the number of full blinks, the number of incomplete blinks, and the number of eye rubbings. Based on this driver state dataset, the processing module 120 determines the driver's eye state and, based on the driver's eye state, determines vehicle control information and sends the control information to the control module 130. The control module 130 adjusts the vehicle's in-vehicle environment information according to the received control parameters.

[0030] For example, the processing module 120 can also be used to acquire the vehicle's historical trip data and the driver's historical eye data, and determine the driver's total driving time in the first time period based on the historical trip data. Based on the total driving time and historical eye data, the probability of the driver experiencing eye discomfort is determined. Finally, the driver's eye detection strategy is determined based on the probability, and the eye detection strategy is sent to the control module 130, so that the control module 130 can perform trip planning for the vehicle based on the eye detection strategy.

[0031] Figure 1 The system in [the document] can be used to implement the following Figure 2 For the driver state detection method described, please refer to [link / reference]. Figure 2 , Figure 2 This application illustrates a driver state detection method according to an embodiment of the present application. In a specific embodiment, this driver state detection method can be applied to, for example... Figure 5 The driver status detection device 500 and the vehicle 1100 equipped with the driver status detection device 500 are shown. Figure 13 The specific process of this embodiment will be described below. Of course, it is understood that this method can be executed by an electronic device with computing power, such as a vehicle-mounted server, a cloud server, or other processors. The following will focus on... Figure 2 The process shown is described in detail. The driver's state detection method may specifically include the following steps 210-230.

[0032] Step 210: Obtain multiple frames of facial images of the driver within the first time period, and determine the driver state dataset based on the multiple frames of facial images.

[0033] As an alternative approach, drivers need to continuously monitor the driving environment and vehicle status while driving. If eye discomfort symptoms occur due to excessive eye strain, potentially leading to a dangerous accident, it can be determined by identifying the driver's facial image data within the cockpit. Drivers with dry eye syndrome are more prone to eye discomfort while driving compared to healthy individuals; therefore, timely identification of these symptoms is crucial for driver safety.

[0034] Optionally, when a driver experiences eye discomfort (such as dry eye syndrome), in order to alleviate the discomfort caused by dry eye, they may exhibit compensatory behavioral responses such as abnormal blinking frequency, eye rubbing, incomplete eye closure, abnormal blinking duration, and squeezing blinks. These responses, in turn, can exacerbate the discomfort and damage to the ocular surface. Therefore, multi-frame facial image data of the driver can be acquired within a first time period. By recognizing these multi-frame facial image data, a driver state dataset can be determined.

[0035] Optionally, the driver status dataset may include multiple eye aspect ratios, multiple mouth aspect ratios, and the number of times the driver rubs their eyes, corresponding to multiple frames of the driver's face images.

[0036] In some optional embodiments, image recognition can be performed on multiple frames of facial image data to determine key eye and mouth features in each frame. The aspect ratio of the eyes and mouth can then be determined based on these key eye features. Optionally, for a single eye, the inner corner, outer corner, upper eyelid line feature points, lower eyelid line feature points, and lower eyelid line feature points can be determined by recognizing facial image data. The aspect ratio of the eye is then calculated based on these determined feature points, and finally, the aspect ratio of the eye corresponding to each frame of facial image data is determined based on the aspect ratios of the left and right eyes.

[0037] Optionally, for a single eye, it can be achieved through a formula. To calculate the corresponding aspect ratio of a single eye, where, For the inner corner of the eye, Feature points along the upper eyelid line, Feature points along the upper eyelid line, For the outer corner of the eye, Feature points along the lower eyelid line, Feature points along the lower eyelid line, || || represents the Euclidean distance between two feature points. Characterizes the distance the eyes open and close in the vertical direction. The width of the eye in the horizontal direction is used for normalization to eliminate the influence of changes in facial distance. Optionally, after determining the aspect ratio of the left and right eyes, the arithmetic mean of the aspect ratios of the left and right eyes is determined to obtain the eye aspect ratio corresponding to each frame of facial image data.

[0038] Optionally, facial image data can be used to identify the left corner of the mouth, the right corner of the mouth, the midpoint of the top of the upper lip, the midpoint of the top of the lower lip, the midpoint of the bottom of the upper lip, and the midpoint of the bottom of the lower lip. Then, the horizontal distance of the mouth is determined based on the left and right corners of the mouth, the first vertical distance is determined based on the midpoint of the top of the upper lip and the midpoint of the bottom of the lower lip, and the second vertical distance is determined based on the midpoint of the bottom of the upper lip and the midpoint of the top of the lower lip. Finally, the aspect ratio of the mouth is determined based on the horizontal distance, the first vertical distance, and the second vertical distance.

[0039] Step 220: Determine at least one of the driver's squeeze blink count, full blink count, incomplete blink count, and eye rubbing count based on the driver state dataset.

[0040] As an alternative approach, the driver status dataset may include the driver's eye feature data, mouth feature data, and the driver's hand and face related feature data. This allows for the determination of features used to determine whether the driver is experiencing eye discomfort based on the driver status dataset, and further, the determination of whether the driver is in a state of eye discomfort based on these features.

[0041] In some alternative embodiments, during a complete blink, the upper and lower eyelids fully contact and completely cover the entire cornea, while during a partial blink, the eyelids do not close completely, leaving the lower or central cornea uncovered. Therefore, the eye aspect ratio is lowest when the eyes are fully closed, and while it decreases when the eyes are partially closed, it remains higher than when the eyes are fully closed. Furthermore, the number of complete and incomplete blinks by the driver can be determined by comparing multiple eye aspect ratio values.

[0042] In some alternative embodiments, a squeeze blink is a blinking action that is more forceful and has a greater amplitude than a normal blink, typically accompanied by strong contraction of the orbicularis oculi muscle and downward pressure of the brow. Furthermore, the aspect ratio of the eye during a squeeze blink is less than that during a full blink; therefore, a squeeze blink can be identified by comparing the aspect ratio values.

[0043] Optionally, if the driver's eye aspect ratio is significantly lower than that of a full blink, the driver may be experiencing fatigue or drowsiness. To ensure the accuracy of the determined number of squeeze blinks, the mouth aspect ratio can be considered to rule out fatigue or drowsiness as the cause of the significantly lower eye aspect ratio. Optionally, when a driver is fatigued or drowsy, they may yawn while blinking forcefully and with a greater amplitude than normal eye closure. Therefore, if the mouth aspect ratio is determined to be greater than a certain value, it can be determined that the driver's forceful and exaggerated blinking is not due to dry eye syndrome.

[0044] Optionally, a combination of the Pipnet facial landmark model and the YOLOv5n Hands hand detection model can be used to identify multiple frames of facial image data to determine the number of times the driver rubs their eyes within the first time period. The Pipnet facial landmark model and the YOLOv5n Hands hand detection model are used to identify multiple frames of facial image data, thereby extracting key points of the driver's eye region (left and right eye contours) and hand key points (fingertips, knuckles, joints, etc.) in real time. Then, the normalized two-dimensional Euclidean distance between the hand key points and the eye center point is calculated. If the distance is less than a set threshold (e.g., 0.03), the corresponding frame of facial image data is determined to be in a hand-eye proximity state, and the persistence of this state is verified by a continuous frame counter, thus obtaining the number of consecutive image frames in the multi-frame facial image data that are in a hand-eye proximity state. When the number of consecutive image frames in the hand-eye proximity state is not less than a set frame threshold, it is determined as a valid eye-rubbing action, avoiding instantaneous interference, thereby determining the number of times the driver rubs their eyes in the multi-frame facial images. Specifically, when the image recognition results indicate that the hand is positioned significantly higher than the forehead area (i.e., the vertical coordinate of the hand is lower than the forehead baseline) or the hand is positioned extremely close to the tip of the nose and far from the eyes, it is determined to be an invalid eye-rubbing action.

[0045] Step 230: Determine the driver's eye state based on at least one of the number of squeeze blinks, the number of complete blinks, the number of incomplete blinks, and the number of eye rubbings.

[0046] As an alternative approach, since complete blinking, incomplete blinking, squeezing blinking, and eye rubbing are all compensatory behaviors caused by dry eye syndrome, it is possible to determine whether a driver is experiencing eye discomfort due to dry eye syndrome based on at least one of the following characteristics: the number of complete blinks, the number of incomplete blinks, the number of squeezing blinks, and the number of eye rubbing.

[0047] In some alternative embodiments, the driver's eye condition can be determined by comparing the number of squeeze blinks, the number of complete blinks, the number of incomplete blinks, and the number of eye rubbings with their respective characteristic thresholds. For example, if the number of squeeze blinks within a first duration is greater than or equal to a first threshold, the driver is determined to be in a state of eye discomfort; or if the number of complete blinks within a first duration is greater than or equal to a second threshold, the driver is determined to be in a state of eye discomfort.

[0048] Optionally, to ensure the accuracy of determining the driver's eye condition, the driver's eye condition can be determined by combining any two or more of the following: the number of squeeze blinks, the number of complete blinks, the number of incomplete blinks, and the number of eye rubbing.

[0049] Optionally, since drivers can relieve eye discomfort by blinking, if a driver experiences eye discomfort, the driver's eye condition can be determined by determining the percentage of complete blinks out of all blinks. Alternatively, the total number of blinks can be determined first based on the number of complete blinks and the number of incomplete blinks, and then the percentage can be determined by calculating the ratio between the number of complete blinks and the total number of blinks, thereby determining the driver's eye condition.

[0050] As an alternative approach, the driver's blink frequency within a first duration can be determined by summing the number of complete blinks and the number of incomplete blinks, and then by determining the ratio of the sum to the first duration.

[0051] In other embodiments, the number of blinks and the sum of blinks can be determined first based on the number of complete blinks and the number of incomplete blinks, and then the ratio between the number of incomplete blinks and the sum of blinks can be determined to obtain the percentage of incomplete blinks.

[0052] As an alternative approach, the driver's eye-rubbing frequency within the first duration can be obtained by determining the ratio between the number of eye-rubbings and the first duration, and the driver's squeeze-blink frequency within the first duration can be determined by determining the ratio between the number of squeeze-blinks and the first duration.

[0053] In the embodiments of this application, a driver's state dataset is determined by analyzing multiple frames of facial images of the driver within a first time period. Based on this state dataset, parameters indicating the driver's eye state can be determined, thereby improving the accuracy of determining the driver's eye state. This allows for timely detection of whether the driver is experiencing persistent eye discomfort, thus preventing accidents from occurring.

[0054] Please see Figure 3 , Figure 3This application illustrates a driver state detection method according to an embodiment of the present application. The following will focus on... Figure 3 The process shown is described in detail. The driver's state detection method may specifically include the following steps 310-340.

[0055] Step 310: Obtain multiple frames of facial images of the driver within the first time period, and determine the driver state dataset based on the multiple frames of facial images.

[0056] Step 320: Determine at least one of the driver's squeeze blink count, full blink count, incomplete blink count, and eye rubbing count based on the driver state dataset.

[0057] Step 330: Determine the driver's eye state based on at least one of the number of squeeze blinks, the number of complete blinks, the number of incomplete blinks, and the number of eye rubbings.

[0058] The specific steps of steps 310-330 can be found in steps 210-230, and will not be repeated here.

[0059] Step 340: If the driver's eye condition is eye discomfort, then adjust the vehicle's interior environment information; the interior environment information includes the vehicle's humidity, or the interior environment information includes the vehicle's humidity and one or more of the following: vehicle temperature, air conditioning direction, air conditioning speed, in-vehicle display parameters, or in-vehicle lighting parameters.

[0060] As an alternative approach, when it is determined that the driver's eye condition is eye discomfort, in order to avoid safety accidents caused by the driver's eye discomfort, the in-vehicle environment can be adjusted to alleviate the driver's eye discomfort symptoms to a certain extent, thereby ensuring the driver's driving safety and improving the driver's driving experience.

[0061] Alternatively, since the in-vehicle environment can affect the driver's eye discomfort to some extent, the in-vehicle environment can be changed by adjusting different parameters inside the vehicle, thereby alleviating the driver's eye discomfort.

[0062] Optionally, since the humidity of the environment has a significant impact on the driver's eye experience, when it is determined that the driver's eye condition is uncomfortable, the driver's eye discomfort can be alleviated by adjusting the humidity inside the vehicle. This ensures that the driver can alleviate the symptoms of eye discomfort to some extent even while driving, and also ensures the driver's driving safety.

[0063] Optionally, since factors such as the vehicle's interior temperature, air conditioning direction and speed, in-vehicle display parameters, and interior lighting parameters can also affect the driver's eye condition to some extent, when it is determined that the driver is experiencing eye discomfort, various parameters of the driver's in-vehicle environment can be adjusted to alleviate the driver's eye discomfort and improve the driver's experience.

[0064] In some alternative embodiments, the brightness and angle of light entering the driver's eyes can be controlled by adjusting the vehicle's sun visor, thereby alleviating the driver's eye discomfort and improving the driver's experience.

[0065] In some embodiments, step 340 includes: when the driver is experiencing eye discomfort and dry eye symptoms, controlling the vehicle's humidity control system to adjust the humidity inside the vehicle, wherein the humidity control system is used to adjust the humidification area inside the vehicle to match the position of the driver's eyes in the cabin.

[0066] As an alternative approach, since dry eye is primarily caused by tear film instability or ocular surface microenvironment imbalance due to abnormalities in tear quality, quantity, and hydrodynamics, the discomfort caused by dry eye is usually relieved by replenishing tear reserves and reducing tear evaporation. However, traditional dry eye interventions (such as eye drops, warm compresses, and medicated fumigation) require specific operating conditions and fixed body positions, which directly conflict with the driving safety requirements and operational constraints of vehicles, and therefore cannot be directly applied in driving scenarios. Therefore, alleviating dry eye can be achieved by changing the humidity inside the vehicle.

[0067] Optionally, after obtaining the vehicle's interior humidity information, humidity adjustment information can be determined based on the interior humidity information. This humidity adjustment information can then be used to control the humidity control system in the vehicle, thereby adjusting the humidity inside the vehicle.

[0068] In some alternative embodiments, since the humidity inside the vehicle is directly related to the temperature inside the vehicle, the air conditioning fan speed, and the humidification power of the vehicle humidifier, the temperature and fan speed parameters of the vehicle air conditioning and the humidification power parameters of the vehicle humidifier can be determined based on the humidity information inside the vehicle. In this way, the corresponding vehicle air conditioning and vehicle humidifier can be controlled based on the temperature and fan speed parameters of the vehicle air conditioning and the humidification power parameters of the vehicle humidifier to adjust the humidity inside the vehicle.

[0069] In this embodiment, after determining the driver's eye condition, when the driver is experiencing eye discomfort, the vehicle's interior environment can be adjusted. This can be done by adjusting the vehicle's humidity, temperature, air conditioning direction and speed, in-vehicle display parameters, or in-vehicle lighting parameters to adaptively alleviate the driver's eye discomfort and improve the driver's driving experience.

[0070] Please see Figure 4 , Figure 4 This application illustrates a driver state detection method according to an embodiment of the present application. The following will focus on... Figure 4 The process shown is described in detail. The driver state dataset includes at least one or more of the following combinations: eye aspect ratio, mouth aspect ratio, number of eye rubbing parameters, blinking frequency, or distance between the hand and the eye. The driver state detection method may specifically include the following steps 410-460.

[0071] Step 410: Obtain multiple frames of facial images of the driver within the first time period, and determine the driver state dataset based on the multiple frames of facial images.

[0072] Step 420: If the driver state dataset includes the aspect ratio of the eyes, determine the number of squeeze blinks based on the multi-frame face images and the aspect ratio of the eyes in each face image.

[0073] As an alternative, squeeze blinking is a blinking motion that is more forceful and larger than normal eye closing, usually accompanied by strong contraction of the orbicularis oculi muscle and downward pressure of the brow. Consequently, the aspect ratio of the eye during squeeze blinking is less than that of a full blink. Therefore, squeeze blinking can be identified by comparing the aspect ratio of the eyes.

[0074] Optionally, if the driver's eye aspect ratio is significantly lower than that of a full blink, the driver may be experiencing fatigue or drowsiness. To ensure the accuracy of the determined number of squeeze blinks, the mouth aspect ratio can be considered to rule out fatigue or drowsiness as the cause of the significantly lower eye aspect ratio. Optionally, when a driver is fatigued or drowsy, they may yawn while blinking forcefully and with a greater amplitude than normal eye closure. Therefore, if the mouth aspect ratio is determined to be greater than a certain value, it can be determined that the driver's forceful and exaggerated blinking is not due to dry eye syndrome.

[0075] In some embodiments, the squeeze blink is determined based on preset conditions corresponding to at least one of the following parameters: total blink duration, minimum eye aspect ratio, cumulative tremor intensity of changes in eye aspect ratio, degree of contraction of the periorbital region, and mouth aspect ratio; after step 420, the method further includes: performing a squeeze blink recognition operation on the face image; when the features of the face image meet the first condition, it is determined that the face image contains a squeeze blink action.

[0076] As an alternative approach, since squeeze blinking is an involuntary, forceful state of closing the eyelids of both (or one) eyes, and the duration of eye closure during squeeze blinking is longer, the number of squeeze blinks within the first duration can be determined by combining the blink duration and the aspect ratio of the eyes corresponding to the multi-frame continuous facial image data.

[0077] In some alternative embodiments, since the eye aspect ratio is consistently below a baseline threshold during squeeze blinks, multiple possible blink intervals with eye aspect ratios consistently below a first threshold can be identified from multiple eye aspect ratios. These multiple possible blink intervals are then marked as potential blink intervals, thereby determining whether squeeze blinks exist in the corresponding potential blink intervals.

[0078] Optionally, when a squeeze blink occurs, the total duration of the blink must be within a reasonable range, and the minimum eye aspect ratio within the potential blink interval must be lower than a more stringent depth threshold. Therefore, the presence of a squeeze blink within a given blink interval can be determined based on the target duration of each possible blink interval and the minimum eye aspect ratio within the target duration.

[0079] As an alternative approach, static features such as the intensity of crow's feet (outer canthal wrinkles), fine lines under the eyes, under-eye bags, and eyebrow descent distance can be determined for each frame of facial image data based on multiple frames of facial image data. Dynamic features are then determined based on multiple eye aspect ratios. Finally, the static and dynamic features are combined to comprehensively determine the degree of periorbital contraction of the driver within a target duration. Optionally, eyelid closure strength and speed can be determined based on multiple eye aspect ratios within the target duration. Specifically, eyelid closure strength can be determined by identifying the minimum eye aspect ratio within the target duration, and eyelid closure speed can be determined based on the downward slope of the eye aspect ratio in consecutive frames of facial image data.

[0080] As an alternative approach, cumulative jitter intensity is used to indicate the overall severity of involuntary, rapid, high-frequency eye tremors (distinguished from normal blinking), while the differential eye aspect ratio between adjacent frames reflects the instantaneous speed and direction changes of eyelid movement. Jitter (especially high-frequency, involuntary tremors) causes rapid, large fluctuations in the differential sequence. Therefore, by summing (integrating) the absolute values ​​or squares of the differential sequence, the total intensity of eyelid tremors over a period of time can be quantified. Furthermore, the differential eye aspect ratio between adjacent frames can be determined first based on multiple eye aspect ratios, and then the cumulative jitter intensity of multiple eye aspect ratios can be determined based on this differential eye aspect ratio.

[0081] In some alternative embodiments, after determining the differential eye aspect ratio between adjacent frames, the sum of squares of the differential eye aspect ratios for all eye aspect ratios is determined to determine the cumulative jitter intensity of multiple eye aspect ratios.

[0082] In some embodiments, the first condition includes: the target duration of blinking is within the duration range, the minimum eye aspect ratio is the eye aspect ratio threshold, the target degree of contraction of the periorbital region is within the target degree of contraction range, the change amplitude of the cumulative tremor intensity is within the target amplitude range, and the facial state indicated by the mouth aspect ratio is that the driver has not performed the target action.

[0083] As an alternative approach, since squeeze blinking involves very forceful eyelid closure, often accompanied by significant periocular skin folds, and its blink duration is longer than that of a normal blink, it can be determined that during squeeze blinking, the corresponding eye aspect ratio is very small, the blink duration is within a reasonable range, and the corresponding cumulative tremor intensity and periocular contraction are also within reasonable ranges, without events such as yawning. Therefore, duration thresholds, eye aspect ratio thresholds, periocular contraction thresholds, and cumulative tremor intensity thresholds can be pre-determined. Based on these thresholds, the target duration, minimum eye aspect ratio, periocular contraction, cumulative tremor intensity, and mouth aspect ratio can be determined to ensure they meet the corresponding conditions.

[0084] In some optional embodiments, in order to accurately determine the number of squeeze blinks, when the target duration, minimum eye aspect ratio, degree of periorbital contraction, cumulative tremor intensity and mouth aspect ratio meet the corresponding conditions, it is also necessary to determine whether there are interfering actions such as yawning based on the mouth aspect ratio. In this way, when it is determined that there are no interfering actions such as yawning based on the mouth aspect ratio, it is determined that there are squeeze blinks in the corresponding target blink interval.

[0085] In some embodiments, the preset conditions include: the target duration is within the target duration range, the minimum eye aspect ratio is less than a second threshold, the cumulative tremor intensity is within the target tremor intensity range, the periorbital contraction degree is within the target contraction degree range, and the mouth aspect ratio indicates that the driver has not performed the target action, wherein the second threshold is less than the first threshold.

[0086] As an optional approach, the target duration can be 6–8 to 15–20 frames of facial image data, approximately 200–270 ms to 500–700 ms. The second threshold can be any value between 0.13 and 0.2. The target jitter intensity can be between 10.0 and 15.0, the target contraction degree can be between 0.70 and 0.8, and the target action can be yawning. Optionally, if the driver yawns, the corresponding mouth aspect ratio increases. Therefore, a mouth aspect ratio threshold can be preset to determine whether the driver has yawned or performed other actions by determining the mouth aspect ratio and the mouth aspect ratio threshold.

[0087] Step 430: Obtain the baseline value of the eye aspect ratio, and determine the closed eye threshold and the incomplete blinking threshold based on the baseline value; the incomplete blinking threshold is higher than the closed eye threshold.

[0088] As an alternative approach, in order to determine the number of complete blinks and incomplete blinks of the driver within a first duration based on multiple eye aspect ratios determined from multiple facial image data, it is possible to determine whether the driver is experiencing eye discomfort based on the number of complete blinks and incomplete blinks.

[0089] In some optional embodiments, in order to accurately determine the number of complete blinks and the number of incomplete blinks and to adapt to the different eye physiological characteristics of drivers, a preset number of reference eye aspect ratios can be collected in advance within a certain period of time when the driver is awake. Based on this preset number of reference eye aspect ratios, the most frequently occurring eye aspect ratio can be determined as the benchmark value for normal eye opening. This allows the eye closing threshold and the incomplete blinking threshold to be determined based on the benchmark value.

[0090] Optionally, after obtaining the baseline value, a first weight and a second weight can be obtained. The eye-closing threshold can be determined based on the first weight and the baseline value, and the incomplete blinking threshold can be determined based on the second weight and the baseline value. For example, the eye-closing threshold can be Th1 = 0.6 × B, and the incomplete eye-closing threshold can be Th2 = 0.75 × B, where B is the baseline value, 0.6 is the first weight, and 0.75 is the second weight.

[0091] Step 440: Compare the aspect ratio of the eyes in each frame of the face image with the closed-eye threshold to determine the number of blinks.

[0092] As an alternative approach, since the upper and lower eyelids fully contact and completely cover the cornea during a complete blink, the vertical opening distance of the eye is reduced, resulting in a decreased aspect ratio. Therefore, it is necessary to determine whether a driver is blinking within a complete blink waveform. Thus, a complete blink waveform can be determined based on multiple aspect ratios, i.e., a complete blink waveform can be identified, and then the minimum aspect ratio within the complete blink waveform can be used to determine whether each complete blink waveform corresponds to a single complete blink.

[0093] In some alternative embodiments, if the eye aspect ratio value corresponding to consecutive frames of facial image data decreases and then increases among multiple eye aspect ratios, it can be determined as a complete blink waveform, thereby enabling the determination of whether a complete blink exists based on the complete blink waveform.

[0094] Optionally, since the aspect ratio of the eye is small when a complete blink occurs, the minimum aspect ratio of the eye can be determined in the complete blink waveform, and the presence of a complete blink can be determined based on the target aspect ratio.

[0095] Optionally, within a complete waveform, the minimum eye aspect ratio is compared to the corresponding eye-closing threshold. If the minimum eye aspect ratio is less than the corresponding eye-closing threshold, then a valid complete blink can be determined within the complete waveform. In this way, the number of blinks can be determined by analyzing the changes in eye aspect ratio values ​​and the minimum eye aspect ratio in multi-frame facial image data.

[0096] And / or, in step 450, compare the aspect ratio of the eyes in each frame of the face image, the closed eye threshold, and the incomplete blink threshold to determine the number of incomplete blinks.

[0097] As an alternative approach, when incomplete blinking occurs, the eyelids do not close completely, and the vertical opening and closing distance of the eyes is smaller compared to when the eyes are open, but larger compared to when blinking completely. This results in an eye aspect ratio that is greater than that when the eyes are fully closed but smaller than that when the eyes are open. Therefore, the number of incomplete blinks can be determined by the eye-closing threshold, the incomplete blinking threshold, and the target eye aspect ratio in each complete blink waveform.

[0098] In some optional embodiments, if the smallest eye aspect ratio in a complete blink waveform is greater than the closed-eye threshold and less than or equal to the incomplete blink threshold, it can be determined that there is one incomplete blink in the corresponding complete blink waveform, thereby enabling the determination of the corresponding number of incomplete blinks based on multi-frame facial image data.

[0099] Optionally, to avoid noise affecting the determined number of incomplete blinks, if the duration of the incomplete blink action exceeds the micro-vibration threshold in the corresponding complete waveform, then the action is determined to be an incomplete blink action.

[0100] Step 460: Determine the driver's eye state based on at least one of the number of squeeze blinks, the number of complete blinks, the number of incomplete blinks, and the number of eye rubbings.

[0101] In some embodiments, step 460 further includes: determining the score corresponding to each number of times, and determining the driver's eye detection score based on the score corresponding to each number of times; if the eye detection score reaches a score threshold, then determining that the eye condition indicates that the driver is in an eye discomfort state.

[0102] As an alternative approach, in order to accurately determine whether the driver experienced eye discomfort due to dry eye syndrome within the first time period, an indicator score table can be obtained first, and then the score corresponding to each blink frequency can be determined in the indicator score table to determine the driver's eye detection score.

[0103] In some optional embodiments, multiple different scoring ranges can be pre-set based on different blink rate ranges, incomplete blink rate ranges, eye rubbing frequency ranges, and squeeze blink frequency ranges to obtain an index scoring table, thereby determining the different scores corresponding to different parameters in the index scoring table. Optionally, the index scoring table is shown in Table 1 below.

[0104]

[0105] Table 1 Indicator Score Table As an alternative approach, after determining the scores for each number of attempts, the driver's eye detection score can be obtained by weighted summation of the scores for each number of attempts.

[0106] In some optional embodiments, weights corresponding to different parameters can be pre-set, thereby determining the driver's eye detection score based on these weights. Specifically, the first weight corresponding to blink frequency can be 0.3, the second weight corresponding to the percentage of incomplete blinks can be 0.30, the third weight corresponding to eye rubbing frequency can be 0.20, and the fourth weight corresponding to squeezing blink frequency can be 0.20. The weights corresponding to different parameters can be set according to actual needs; this is merely an example.

[0107] Optionally, to accurately determine the driver's eye condition (e.g., whether the driver's eye discomfort is due to dry eye syndrome), a pre-set score threshold can be configured to indicate whether the driver is experiencing eye discomfort due to factors such as blink frequency, the percentage of incomplete blinks, eye rubbing frequency, and squeezing blink frequency. Therefore, when the driver's eye detection score reaches the score threshold, it can be determined that the driver is experiencing eye discomfort.

[0108] For a detailed description of step 410, please refer to step 210, which will not be repeated here.

[0109] In this embodiment, the number of squeeze blinks is determined based on the aspect ratio of the eyes, and the eye closure threshold and the incomplete eye closure threshold are determined based on the obtained reference value of the eye aspect ratio. This allows the number of blinks and the number of no blinks to be determined based on the incomplete eye closure threshold and the complete eye closure threshold. This allows the driver's eye condition to be determined based on at least one of the squeeze blinks, complete blinks, incomplete blinks, and eye rubbing, thereby improving the accuracy of determining whether the driver is in a state of eye discomfort.

[0110] The above embodiments describe in detail the driver state detection method provided in this application. In other embodiments, this application also provides a driver state detection device. Figure 5 This is a block diagram of a driver status detection device according to an embodiment of this application, such as... Figure 5 As shown, the driver's state detection device 500 includes a first acquisition module 510, a first determination module 520, and a second determination module 530.

[0111] The first acquisition module 510 is used to acquire multiple frames of facial images of the driver within a first time period, and determine a driver state dataset based on the multiple frames of facial images; the first determination module 520 is used to determine at least one of the driver's squeeze blink count, full blink count, incomplete blink count, and eye rubbing count based on the driver state dataset; the second determination module 530 is used to determine the driver's eye state based on at least one of the squeeze blink count, full blink count, incomplete blink count, and eye rubbing count.

[0112] In some embodiments, the driver's state detection device 500 further includes: an adjustment module, used to adjust the vehicle's interior environment information if the driver's eye condition is eye discomfort; the interior environment information includes interior humidity, or the interior environment information includes interior humidity and one or more of the following: interior temperature, air conditioning direction, air conditioning speed, interior display parameters, or interior lighting parameters.

[0113] In some embodiments, the adjustment module includes a control submodule, configured to control the vehicle's humidity control system to adjust the humidity inside the vehicle when the driver is experiencing eye discomfort and dry eye symptoms, wherein the humidity control system is configured to adjust the humidification area inside the vehicle to match the position of the driver's eyes in the cabin.

[0114] In some embodiments, the driver state dataset includes at least one or more combinations of the following: eye aspect ratio, mouth aspect ratio, number of eye rubbing parameters, blinking frequency, or distance between the hand and the eye.

[0115] In some embodiments, where the driver state dataset includes an eye aspect ratio, the first determining module 520 includes: a squeeze blink count determining submodule, configured to determine the squeeze blink count based on the multi-frame facial images and the eye aspect ratio of each frame of facial images; and / or, a threshold determining submodule, configured to obtain a reference value of the eye aspect ratio and determine a closed-eye threshold and an incomplete blink threshold based on the reference value; wherein the incomplete blink threshold is higher than the closed-eye threshold; a blink count determining submodule, configured to compare the eye aspect ratio of each frame of facial images with the closed-eye threshold to determine the blink count; and an incomplete blink count determining submodule, configured to, and / or, compare the eye aspect ratio of each frame of facial images, the closed-eye threshold, and the incomplete blink threshold to determine the incomplete blink count.

[0116] In some embodiments, the squeeze blink is determined based on preset conditions corresponding to at least one of the following parameters: total blink duration, minimum eye aspect ratio, cumulative tremor intensity of changes in eye aspect ratio, degree of contraction in the periorbital region, and mouth aspect ratio; the first determining module 520 further includes: an execution submodule, used to perform a squeeze blink recognition operation on the face image; and a squeeze blink action determining submodule, used to determine that the face image contains a squeeze blink action when the features of the face image meet a first condition.

[0117] In some embodiments, the first condition includes: the target duration of blinking is within the duration range, the minimum eye aspect ratio is the eye aspect ratio threshold, the target degree of contraction of the periorbital region is within the target degree of contraction range, the change amplitude of the cumulative tremor intensity is within the target amplitude range, and the facial state indicated by the mouth aspect ratio is that the driver has not performed the target action.

[0118] Figure 1 The system can also be used to implement the following Figure 6 For the vehicle control method described, please refer to [link / reference]. Figure 6 , Figure 6This application illustrates a vehicle control method according to an embodiment of the present application. In a specific embodiment, the vehicle control method can be applied to, for example... Figure 10 The vehicle control device 1000 and the vehicle 1100 equipped with the vehicle control device 1000 are shown. Figure 13 The specific process of this embodiment will be described below. Of course, it is understood that this method can be executed by an electronic device with computing power, such as a vehicle-mounted server, a cloud server, or other processors. The following will focus on... Figure 6 The process shown is described in detail, and the vehicle control method may specifically include the following steps 610-630.

[0119] Step 610: Obtain the vehicle's historical travel data and the driver's historical eye data, and determine the driver's total driving time in the first time period based on the historical travel data.

[0120] As an alternative approach, since people with dry eye syndrome are far more likely to experience eye discomfort than those without, in order to ensure that people with dry eye syndrome receive early warnings of eye discomfort while driving and to avoid wasting resources by monitoring the eyes of healthy people, risk assessments can be conducted on different drivers in advance. This allows for continuous monitoring of high-risk drivers while eliminating the need for eye monitoring of low-risk drivers.

[0121] In some alternative embodiments, to determine whether a driver is prone to eye discomfort due to dry eye syndrome, analysis can be performed based on the driver's historical driving data and historical eye data to determine the corresponding eye detection strategy. Therefore, the driver's driving duration within a first time period can be determined by acquiring the driver's historical travel data.

[0122] Optionally, to ensure the accuracy of driver risk assessment, the driver's driving time can be determined by obtaining the vehicle's historical mileage data over the past 30 days. Specifically, the number of days the same driver drove the vehicle within those 30 days can be determined using the vehicle's historical mileage data, thus obtaining the driver's driving time.

[0123] Step 620: Determine the probability of the driver experiencing eye discomfort based on the total driving time and the historical eye data; the probability is determined by at least one of the number of squeeze blinks, full blinks, incomplete blinks, and eye rubbings indicated by the historical eye data.

[0124] As an alternative approach, to accurately assess the risk of eye discomfort in drivers, one can first analyze the frequency of eye discomfort experienced by drivers each day while driving. Based on the cumulative frequency of eye discomfort each day, the probability of eye discomfort in drivers can be obtained.

[0125] Optionally, the frequency of eye discomfort can be determined by analyzing historical eye data to identify at least one of the following during each driving session: the number of complete blinks, the number of incomplete blinks, the number of squeeze blinks, and the number of eye rubbing.

[0126] Step 630: Determine the driver's eye detection strategy based on the probability, and execute the vehicle's trip planning based on the eye detection strategy; the trip planning includes vehicle control behavior when the vehicle is in a stopable area or a non-stopable area.

[0127] As an alternative approach, different eye detection strategies can be pre-set for different probability ranges. This allows for vehicle trip planning based on the eye detection strategies and the driver's eye condition. This enables the driver to anticipate and prevent potential eye discomfort, thereby improving the driver's experience.

[0128] Optionally, after determining the probability of the driver experiencing eye discomfort, the corresponding eye detection strategy can be determined based on the probability range in which the driver experiences eye discomfort, with different eye detection strategies corresponding to different probability ranges.

[0129] As an alternative approach, since vehicles cannot stop arbitrarily during driving, it is possible to first determine whether the vehicle is in a non-parking zone based on its real-time location. If it is determined to be in a non-parking zone, the remaining driving time corresponding to the parking zone is first determined. Then, a prompt message is generated based on the remaining driving time to remind the driver of the remaining road length and remaining driving time. The prompt message is also used to remind the driver again after leaving the non-parking zone.

[0130] In some optional embodiments, if the real-time location is within a designated parking area, route planning is performed based on this area to obtain a first route. This first route guides the vehicle into the parking area, ensuring that the user can intervene immediately within the designated parking area. Optionally, the first route may also include the driver's driving time. For example, the first route may remind the user that they have been driving for X minutes and suggest using eye drops in a safe area to prevent discomfort.

[0131] As an alternative approach, if a non-stopping area exceeding a preset time interval is detected during the driver's current journey, a voice reminder will be given before entering the non-stopping area. This aims to prevent the driver from experiencing sudden eye discomfort in the non-stopping area, potentially leading to a safety accident. Optionally, this voice reminder can be determined based on the path length of the non-stopping area and the vehicle's estimated travel time. For example: "Entering a non-stopping area of ​​B kilometers in A minutes, it is recommended to use eye drops in advance to prevent discomfort."

[0132] In the embodiments of this application, mechanical energy analysis can be performed based on the historical travel data of the vehicle and the historical eye data of the driver to determine the frequency of eye discomfort experienced by the driver during the total driving time. Based on this probability, the driver's eye detection strategy can be determined, and then the vehicle's travel planning can be executed to improve the driver's driving experience.

[0133] Please see Figure 7 , Figure 7 This application illustrates a vehicle control method according to an embodiment of the present application. The following will focus on... Figure 7 The process shown is described in detail, and the vehicle control method may specifically include the following steps 710-750.

[0134] Step 710: Obtain the vehicle's historical travel data and the driver's historical eye data, and determine the driver's total driving time in the first time period based on the historical travel data.

[0135] Step 720: Determine the frequency of eye discomfort experienced by the driver during the first time period based on the historical eye data.

[0136] As an alternative approach, one can first analyze the frequency of eye discomfort experienced by the driver each day while driving based on historical eye data, and then determine the probability of the driver experiencing eye discomfort in the first time period based on the frequency of eye discomfort experienced each day.

[0137] Optionally, historical eye data can be analyzed to determine the number of complete blinks, incomplete blinks, squeezing blinks, and eye rubbing during each driving session, thereby determining the frequency of eye discomfort experienced by the driver. This allows for the determination of the probability of eye discomfort experienced by the driver during the first time period based on all historical eye data collected during the first time period.

[0138] Optionally, if the driver experiences eye discomfort each time within the first time period based on historical driving data analysis, the number of times is accumulated, and then the number of times the driver drives within the first time period is determined. Finally, the number of times the driver drives and the number of times eye discomfort occurs within the first time period are determined, thereby determining the frequency of eye discomfort experienced by the driver within the first time period.

[0139] Step 730: If the total driving time is greater than or equal to the first duration, then determine the probability based on the frequency.

[0140] As an alternative approach, when the driver's total driving time within the first time period is greater than or equal to the first time period, the frequency of the driver experiencing eye discomfort within the first time period can be directly determined as the probability of the driver experiencing eye discomfort. For example, if the driver's driving time is 30 days, the frequency of the driver experiencing eye discomfort each day is determined, and the corresponding single-occurrence probability is determined based on the frequency of the driver experiencing eye discomfort each day. Then, the single-occurrence probabilities over 30 days are accumulated to obtain the corresponding probability.

[0141] In some optional embodiments, the initial probability is 0. The probability is calculated based on the driving situation and the frequency of eye discomfort each day over the past 30 days. If the frequency of eye discomfort on a single day is greater than or equal to 1 time / hour, the probability is increased by 3.3%. If the frequency of eye discomfort on a single day is greater than or equal to 0.5 times / hour but less than 1 time / hour, the probability is increased by 2.2%. If the frequency of eye discomfort on a single day is greater than 0 but less than 0.5 times / hour, the probability is increased by 1.1%. If no eye discomfort occurs on a single day, the probability is increased to 0. All probabilities over the 30 days are accumulated to obtain the probability of the driver experiencing eye discomfort.

[0142] Step 740: If the total driving time is greater than or equal to the second duration and less than the first duration, then the probability is determined cumulatively based on the total driving time and the frequency, wherein the second duration is less than the first duration.

[0143] As an optional approach, if the driver's total driving time in the first time period is greater than or equal to the second time period but less than the first time period, then the cumulative reference probability of the driver within the total driving time is first determined, and then the probability of the driver experiencing eye discomfort in the first time period is determined based on the reference probability and the driving time. Optionally, if it is determined that the driver's driving time is less than the second time period, then no risk assessment is performed on the driver, i.e., it is determined that no eye examination will be performed on the driver.

[0144] Optionally, if the driver's driving time is greater than or equal to the second driving time and less than the first driving time, the cumulative frequency of eye discomfort experienced by the driver within the driving time is first determined. Then, an additional probability is determined based on the frequency and the total driving time. Finally, the probability of the driver experiencing eye discomfort within the first driving time period is determined based on the additional probability and the frequency. For example, if the driver's driving time is 20 days, less than 30 days, and greater than 10 days, the frequency is determined to be 20%, the additional probability is determined to be 10%, and the probability is determined to be 30%.

[0145] Step 750: Determine the driver's eye detection strategy based on the probability, and execute the vehicle's trip planning based on the eye detection strategy; the trip planning includes vehicle control behavior when the vehicle is in a stopable area or a non-stopable area.

[0146] For a detailed description of steps 710 and 750, please refer to steps 610 and 630, which will not be repeated here.

[0147] In this embodiment, the probability of the driver experiencing eye discomfort is determined comprehensively based on the total driving time, ensuring the accuracy of the determined eye detection strategy and further improving the driver's driving experience.

[0148] Please see Figure 8 , Figure 8 This application illustrates a vehicle control method according to an embodiment of the present application. The following will focus on... Figure 8 The process shown is described in detail, and the vehicle control method may specifically include the following steps 810-860.

[0149] Step 810: Obtain the vehicle's historical travel data and the driver's historical eye data, and determine the driver's total driving time in the first time period based on the historical travel data.

[0150] Step 820: Determine the probability of the driver experiencing eye discomfort based on the total driving time and the historical eye data; the probability is determined by at least one of the number of squeeze blinks, full blinks, incomplete blinks, and eye rubbings indicated by the historical eye data.

[0151] Step 830: Determine the risk level of the driver experiencing eye discomfort based on the probability.

[0152] As an alternative approach, once the probability is determined, the risk level of eye discomfort for the driver can be directly determined based on the probability. This allows for the determination of the driver's eye detection strategy based on the risk level, and further enables the vehicle's route planning based on the driver's eye detection strategy, thus avoiding the problem of poor driving experience caused by driver eye discomfort.

[0153] Optionally, risk levels can be pre-set for different probability ranges. After determining the probability of a driver experiencing eye discomfort, the corresponding risk level can be determined based on the probability range within which the driver experiences eye discomfort. Optionally, there can be at least three levels: low risk (third risk level), medium risk (second risk level), and high risk (first risk level). A probability of 0% to 33% corresponds to the low risk level, 34% to 66% to the medium risk level, and 67% to 99% to the high risk level.

[0154] Step 840: If the risk level is Level 1 or Level 2, then perform eye detection on the driver and determine the driver's eye detection strategy; wherein the probability of Level 2 indication is less than the probability of Level 1 indication.

[0155] As an alternative approach, when the risk level is determined to be Level 1 or Level 2, it is determined that the probability of the driver experiencing eye discomfort (such as dry eye syndrome) is relatively high. Therefore, continuous monitoring of the driver is required to determine the driver's eye detection strategy as continuous monitoring of the driver while driving the vehicle.

[0156] As an alternative approach, if the risk level is determined to be Level 1 or Level 2 and the driver is confirmed not to be experiencing eye discomfort, in order to improve the driver's experience, the driver can be given advance warnings based on the duration of driving. This allows the driver to make adjustments in advance based on the warnings, thereby alleviating the driver's eye discomfort caused by dry eye syndrome and achieving early intervention.

[0157] In some optional embodiments, since the driver's eye discomfort due to dry eye syndrome only occurs after a certain period of eye use, intervention can be initiated in advance based on the actual driving time and a preset time interval from when the driver starts using their eyes to when eye discomfort occurs. This preset time interval can be a custom time interval set by the driver according to their own circumstances.

[0158] Optionally, if it is determined that the driver is not experiencing eye discomfort based on multi-frame facial image data of the driver within the actual driving time, the remaining time before the driver is about to experience eye discomfort can be determined based on the actual driving time and a preset time interval, and this remaining time can be used as the reminder duration.

[0159] Optionally, the intervention prompt may be a voice prompt indicating that the driver may experience eye discomfort at a specific time, and a corresponding intervention strategy may be suggested, so that the driver can take preventative measures according to the intervention strategy.

[0160] Step 850: If the risk level is level three, then no eye examination is performed on the driver, wherein the probability of the level three indication is less than the probability of the level two indication.

[0161] As an alternative approach, when the risk level is determined to be Level 3, it can be determined that the probability of the corresponding driver experiencing eye discomfort is very small. In order to reduce the computational and resource burden on the vehicle, it can be determined not to perform eye monitoring on the driver.

[0162] Step 860: Perform trip planning for the vehicle according to the eye detection strategy; the trip planning includes vehicle control behavior when the vehicle is in a stopable area or a non-stopable area.

[0163] The specific steps of steps 810-820 and 860 can be found in steps 610-630, and will not be repeated here.

[0164] In this embodiment, the probability of a driver experiencing eye discomfort is determined based on accumulated historical eye data. The risk level is then determined based on the probability of eye discomfort, and the decision on whether to conduct eye monitoring on the driver is based on the risk level. By continuously recording and accumulating historical data, and by analyzing the time series of individual eye discomfort frequency, the system can shift from symptom monitoring to disease probability prediction, supporting dynamic risk level classification and long-term tracking management.

[0165] Please see Figure 9 , Figure 9 This application illustrates a vehicle control method according to an embodiment of the present application. The following will focus on... Figure 9 The process shown is described in detail, and the vehicle control method may specifically include the following steps 910-950.

[0166] Step 910: Obtain the vehicle's historical travel data and the driver's historical eye data, and determine the driver's total driving time in the first time period based on the historical travel data.

[0167] Step 920: Determine the probability of the driver experiencing eye discomfort based on the total driving time and the historical eye data; the probability is determined by at least one of the number of squeeze blinks, full blinks, incomplete blinks, and eye rubbings indicated by the historical eye data.

[0168] For a detailed description of steps 910 and 920, please refer to steps 610 and 620, which will not be repeated here.

[0169] Step 930: Determine the driver's eye detection strategy based on the probability and obtain the vehicle's location information.

[0170] As an alternative approach, after determining the driver's eye detection strategy, in order to ensure the driver's driving experience and driving safety, the vehicle's location information can be obtained first, and then the specific driving plan for the driver can be determined based on the vehicle's location information.

[0171] Step 940: If the location information indicates that the vehicle is in a non-parking area, then determine the travel time of the vehicle from the non-parking area to the parking area, and provide an intervention prompt to the driver based on the travel time, wherein the intervention prompt is used to remind the driver to take precautions to prevent eye discomfort.

[0172] As an alternative approach, since drivers cannot stop arbitrarily while driving, the real-time location of the vehicle can be used to determine whether it is in a non-parking zone. If it is determined that the driver is driving in a non-parking zone, the remaining driving time corresponding to the driving time in the non-parking zone can be determined, i.e., the driving time from the non-parking zone to the parking zone. Then, a first prompt message is generated based on the remaining driving time to remind the driver of the remaining road length and remaining driving time. After leaving the non-parking zone, a reminder is given again to prevent discomfort from eye drops.

[0173] Optionally, if the vehicle's location information indicates that the vehicle is currently in a no-parking zone, a voice reminder can be given to the driver. For example, it is estimated that there are D minutes left before leaving the no-parking zone. Optionally, a voice reminder can be given after the vehicle's location determines that it has left the no-parking zone. For example: "You have left the no-parking zone. It is recommended that you use eye drops in a safe area to relieve eye discomfort."

[0174] Step 950: If the location information indicates that the vehicle is in a parking area, then the driver is given an immediate prompt.

[0175] As an alternative approach, if the vehicle's location information indicates it is within a parking area, a second prompt message can be generated to alert the user to intervene immediately. This second prompt message can then promptly remind the driver to find a safe parking area, enabling proactive intervention. Optionally, the second prompt message can also include the driver's driving time. For example, the second prompt message could be: "You have been driving for X minutes. It is recommended to use eye drops in a safe area to prevent discomfort."

[0176] As an alternative approach, if the driver's current journey detects a non-stopping zone with a duration exceeding a preset time interval, a voice reminder will be given before entering the non-stopping zone. This aims to prevent the driver from experiencing sudden eye discomfort in prohibited areas, thus avoiding potential safety accidents. Optionally, this voice reminder can be determined based on the path length of the non-stopping zone and the vehicle's estimated travel time. For example: "Entering a non-stopping zone of B kilometers in A minutes, it is recommended to use eye drops in advance to prevent discomfort."

[0177] In some embodiments, after step 950, the method further includes: obtaining the vehicle's in-vehicle display parameters; and controlling the in-vehicle display to turn on the eye protection mode based on the in-vehicle display parameters.

[0178] As an alternative approach, since the display parameters of the vehicle's in-vehicle display can also affect the driver's eyes to some extent, in order to alleviate the driver's eye discomfort, the in-vehicle display parameters can be obtained, and then the in-vehicle display can be controlled to turn on the eye protection mode based on the in-vehicle display parameters.

[0179] In some optional embodiments, if the vehicle display parameters indicate that the eye-protection display function is not enabled, the eye-protection display will be automatically enabled. The parameters involved include blue light percentage, PWM dimming frequency, the relationship between brightness and ambient illuminance, the relationship between contrast ratio and ambient illuminance, and color temperature; the specific values ​​depend on the type of display. Optionally, if the vehicle display parameters indicate that the eye-protection display function has been manually disabled, no action will be taken, and the eye-protection display function will not be automatically enabled again before the vehicle's power is cut off.

[0180] In some optional embodiments, after step 950, the method further includes: obtaining in-vehicle humidity information; if it is determined from the in-vehicle humidity information that the humidity inside the vehicle is less than a humidity threshold, and it is determined that the vehicle is not in a long-distance driving state, then determining the vehicle's in-vehicle humidifier parameters and in-vehicle air conditioning parameters.

[0181] As an alternative approach, since the humidity inside a vehicle affects tear production, if the driver is experiencing eye discomfort, reducing tear production can be achieved by adjusting the humidity inside the vehicle, thereby alleviating the driver's eye discomfort. Therefore, after obtaining the vehicle's humidity information, if it is determined that the humidity inside the vehicle is below a certain threshold, it is necessary to increase the humidity level.

[0182] In some alternative embodiments, the humidity inside the vehicle is related to the vehicle's onboard humidifier and air conditioning; therefore, the humidity inside the vehicle can be adjusted by controlling the onboard humidifier and air conditioning.

[0183] Optionally, the vehicle humidifier control parameters may include the humidification level and output power of the vehicle humidifier, and the vehicle air conditioning control parameters may include the air conditioning fan speed, temperature, and airflow direction, so that the humidity inside the vehicle can be adjusted through the vehicle humidifier control parameters and the vehicle air conditioning control parameters.

[0184] In some optional embodiments, after step 950, the method further includes: controlling the vehicle humidifier to humidify the air according to the vehicle humidifier parameters and controlling the vehicle air conditioner to adjust the airflow direction and speed according to the vehicle air conditioner parameters.

[0185] As an optional approach, if the vehicle humidifier parameters indicate that the vehicle humidifier is off and has sufficient water, then the vehicle humidifier will automatically start air driving; if the vehicle humidifier parameters indicate that the vehicle humidifier has been manually turned on, then no intervention will be made, meaning that the vehicle humidifier can only be turned off manually or automatically when the water level is low.

[0186] In some optional embodiments, if the vehicle air conditioning parameters indicate that the vehicle air conditioning is on and the function of avoiding blowing on people is not activated, the air conditioning will be automatically activated to avoid blowing on people; if the vehicle air conditioning parameters indicate that the air conditioning direction has been manually reset by the user, no action will be taken and the air conditioning direction will not be automatically adjusted again before the vehicle's power is cut off; if the vehicle air conditioning parameters indicate that the air conditioning speed has been manually reset by the user, no action will be taken and the air conditioning speed will not be automatically adjusted again before the vehicle's power is cut off.

[0187] In this embodiment, the vehicle's location information can be used to determine whether the vehicle is in a non-parking area or a parking area. When the vehicle is in a non-parking area, the driver can be prompted based on the travel time from the non-parking area to the parking area, thereby further improving the driver's driving experience.

[0188] The above embodiments describe in detail the vehicle control method provided in this application. In other embodiments, this application also provides a vehicle control device 1000. Figure 10 This is a block diagram of a vehicle control device 1000 according to an embodiment of this application, as shown below. Figure 10 As shown, the vehicle control device 1000 includes a second acquisition module 1010, a third determination module 1020, and a control module 1030.

[0189] The second acquisition module 1010 is used to acquire historical travel data of the vehicle and historical eye data of the driver, and determine the total driving time of the driver in a first time period based on the historical travel data; the third determination module 1020 is used to determine the probability of the driver experiencing eye discomfort based on the total driving time and the historical eye data; the probability is determined by at least one of the number of squeeze blinks, the number of complete blinks, the number of incomplete blinks, and the number of eye rubbings indicated by the historical eye data; the control module 1030 is used to determine the driver's eye detection strategy based on the probability, and execute travel planning for the vehicle based on the eye detection strategy; the travel planning includes vehicle control behavior when the vehicle is in a stopable area or a non-stopable area.

[0190] In some embodiments, the third determining module 1020 includes: a frequency determining submodule, configured to determine the frequency of eye discomfort experienced by the driver within a first time period based on the historical eye data; a probability first determining submodule, configured to determine the probability based on the frequency if the total driving time is greater than or equal to a first duration; and a probability second determining submodule, configured to cumulatively determine the probability based on the total driving time and the frequency if the total driving time is greater than or equal to a second duration and less than the first duration, wherein the second duration is less than the first duration.

[0191] In some embodiments, the control module 1030 includes: a risk level determination submodule, configured to determine the risk level of the driver experiencing eye discomfort based on the probability; an eye detection strategy first determination submodule, configured to perform eye detection on the driver and determine the driver's eye detection strategy if the risk level is a first level or a second level; wherein the probability of the second level indication is less than the probability of the first level indication; and an eye detection strategy second determination submodule, configured to not perform eye detection on the driver if the risk level is a third level, wherein the probability of the third level indication is less than the probability of the second level indication.

[0192] In some embodiments, the control module 1030 includes: a location information acquisition submodule, configured to acquire the location information of the vehicle; a first prompting submodule, configured to determine the travel time of the vehicle from the non-parking area to the parking area if the location information indicates that the vehicle is in a non-parking area, and to provide an intervention prompt to the driver based on the travel time, wherein the intervention prompt is used to prompt the driver to take precautions against eye discomfort; and a second prompting submodule, configured to provide an immediate prompt to the driver if the location information indicates that the vehicle is in a parking area.

[0193] Figure 11 A vehicle control method is provided, such as Figure 11 As shown, after the vehicle's infotainment system starts, it first checks whether the user in the driver's seat is a low-risk user. If confirmed, no action is taken. If not, it further checks whether the vehicle is currently using navigation and is on a long-distance trip. If it's not on a long-distance trip, it checks whether the vehicle is on a journey by checking if it's in the parking position. If not in the parking position, it confirms the vehicle is on a journey and continuously performs eye checks. If the eye check results indicate an eye abnormality, it activates the humidifier, turns on the air conditioning's anti-blow function, and activates the screen's eye protection display. Simultaneously, a voice prompt is given to remind the user to find a suitable nearby parking spot to stop and administer eye drops.

[0194] If it is determined that the vehicle is currently using navigation and is on a long-distance drive, the current time T0 and the eye drop interval T1 are obtained. Based on T0 and T1, a voice reminder is given to the user to apply eye drops in advance. When the vehicle is not in park and is in motion, continuous eye monitoring is performed. If the user's eye abnormality is detected, the humidifier is turned on, the air conditioning's anti-blow function is activated, and the screen's eye protection display is turned on. At the same time, a voice reminder is given to remind the user to find a suitable parking spot nearby to stop and apply eye drops.

[0195] If it is determined that the user has not yet experienced any eye abnormalities, the system will determine whether the vehicle is about to enter a no-stopping zone with an estimated travel time exceeding T1 based on the vehicle's real-time location. If it is determined that the vehicle is about to enter a no-stopping zone with an estimated travel time exceeding T1, the system will then determine whether the current time has reached the time points defined by T0 and T1. If the time point has been reached and the vehicle has entered the no-stopping zone, a voice prompt will remind the user of the remaining time for the current travel segment, and a second reminder to use eye drops to prevent eye discomfort will be given after the travel segment ends. Alternatively, if the time point has not been reached, eye monitoring will continue. Alternatively, if the time point has been reached but the vehicle has not entered the no-stopping zone, a voice prompt will remind the user to take preventative measures at a nearby parking spot. Alternatively, if it is determined that the vehicle will not enter a no-stopping zone with an estimated travel time exceeding T1, a voice prompt will remind the user to take preventative measures before entering the no-stopping zone.

[0196] Figure 12 A method for detecting eye discomfort is provided, such as Figure 12 As shown, the real-time data stream of the DMS can be acquired first, and a continuous monitoring loop can be executed to parallelize the visual processing algorithm. Feature extraction can be performed through blink detection algorithm, eye rubbing detection algorithm, and squeeze blink detection algorithm to determine the driver's blink frequency, number of eye rubbings, proportion of incomplete blinks, and number of squeeze blinks during driving. Based on the blink frequency, number of eye rubbings, proportion of incomplete blinks, and number of squeeze blinks, the eye discomfort detection algorithm can determine whether the driver will experience eye discomfort during driving. The detection results are then recorded.

[0197] According to one aspect of the embodiments of this application, a vehicle is also provided, such as Figure 13 As shown, the vehicle 1100 also includes a processor 1110 and one or more memories 1120. The one or more memories 1120 are used to store program instructions executed by the processor 1110. When the processor 1110 executes the program instructions, it implements the above-mentioned driver eye state detection method and / or vehicle control method.

[0198] Furthermore, the processor 1110 may include one or more processing cores. The processor 1110 runs or executes instructions, programs, code sets, or instruction sets stored in the memory 1120, and calls data stored in the memory 1120. Optionally, the processor 1110 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 1110 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor and may be implemented using a separate communication chip.

[0199] According to one aspect of this application, a computer-readable storage medium is also provided, which may be included in the cloud server described in the above embodiments; or it may exist independently and not assembled into the cloud server. The aforementioned computer-readable storage medium carries computer-readable instructions that, when executed by a processor, implement the methods in any of the above embodiments.

[0200] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable storage media can be, for example, but not limited to: electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0201] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0202] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0203] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for detecting the state of a driver, characterized in that, The method includes: Acquire multiple frames of facial images of the driver within a first time period, and determine the driver state dataset based on the multiple frames of facial images; Based on the driver state dataset, determine at least one of the driver's squeeze blink count, full blink count, incomplete blink count, and eye rubbing count; The driver's eye condition is determined based on at least one of the following: the number of squeeze blinks, the number of complete blinks, the number of incomplete blinks, and the number of eye rubbing.

2. The method according to claim 1, characterized in that, The method further includes: If the driver's eye condition is eye discomfort, the vehicle's interior environment information is adjusted; the interior environment information includes the vehicle's humidity, or the interior environment information includes the vehicle's humidity and one or more of the following: vehicle temperature, air conditioning direction, air conditioning speed, in-vehicle display parameters, or in-vehicle lighting parameters.

3. The method according to claim 2, characterized in that, If the driver experiences eye discomfort, the vehicle's interior environment information will be adjusted, including: When the driver experiences eye discomfort and dry eye symptoms, the vehicle's humidity control system is controlled to adjust the humidity inside the vehicle. The humidity control system is used to adjust the humidification area inside the vehicle to match the position of the driver's eyes in the cabin.

4. The method according to claim 1, characterized in that, The driver state dataset includes at least one or more of the following combinations: eye aspect ratio, mouth aspect ratio, number of eye rubbing parameters, blinking frequency, or distance between the hand and the eye.

5. The method according to claim 4, characterized in that, When the driver state dataset includes eye aspect ratio, determining at least one of the driver's squeeze blink count, full blink count, incomplete blink count, and eye rubbing count based on the driver state dataset includes: The number of squeeze blinks is determined based on the multi-frame facial images and the aspect ratio of the eyes in each frame; and / or, Obtain the baseline value of the eye aspect ratio, and determine the closed eye threshold and the incomplete blinking threshold based on the baseline value; the incomplete blinking threshold is higher than the closed eye threshold. The number of blinks is determined by comparing the aspect ratio of the eyes in each frame of the face image with the closed-eye threshold. And / or, compare the aspect ratio of the eyes in each frame of the face image, the closed eye threshold, and the incomplete blink threshold to determine the number of incomplete blinks.

6. The method according to claim 5, characterized in that, The squeeze-type blink is determined based on preset conditions corresponding to at least one of the following parameters: total blink duration, minimum eye aspect ratio, cumulative tremor intensity of changes in eye aspect ratio, degree of contraction in the periorbital area, and mouth aspect ratio. Before determining the number of squeeze blinks, the method further includes: Perform a squeeze-type blink recognition operation on the facial image; When the features of the facial image meet the first condition, it is determined that the facial image contains a squeezing blinking motion.

7. The method according to claim 6, characterized in that, The first condition includes: the target blink duration is within the duration range, the minimum eye aspect ratio is the eye aspect ratio threshold, the target degree of contraction in the periorbital area is within the target degree of contraction, the change range of the cumulative tremor intensity is within the target range, and the facial state indicated by the mouth aspect ratio is that the driver has not performed the target action.

8. The method according to claim 1, characterized in that, Determining the driver's eye state based on at least one of the following: the number of squeeze blinks, the number of complete blinks, the number of incomplete blinks, and the number of eye rubbings, includes: Determine the score corresponding to each number of attempts, and determine the driver's eye detection score based on the score corresponding to each number of attempts; If the eye detection score reaches a threshold, the eye condition indicates that the driver is experiencing eye discomfort.

9. A method for controlling a vehicle, characterized in that, The method includes: The vehicle's historical trip data and the driver's historical eye data are obtained, and the driver's total driving time in the first time period is determined based on the historical trip data. The probability of the driver experiencing eye discomfort is determined based on the total driving time and the historical eye data; the probability is determined by at least one of the driver's squeezing blinks, full blinks, incomplete blinks, and eye rubbing as indicated by the historical eye data. The driver's eye detection strategy is determined based on the probability, and the trip planning for the vehicle is executed according to the eye detection strategy; the trip planning includes vehicle control behavior when the vehicle is in a stopable area or a non-stopable area.

10. The method according to claim 9, characterized in that, The method of determining the probability of the driver experiencing eye discomfort based on the total driving time and the historical eye data includes: The frequency of eye discomfort experienced by the driver during the first time period was determined based on the historical eye data. If the total driving time is greater than or equal to the first duration, then the probability is determined based on the frequency; If the total driving time is greater than or equal to the second driving time and less than the first driving time, the probability is determined cumulatively based on the total driving time and the frequency, wherein the second driving time is less than the first driving time.

11. The method according to claim 9, characterized in that, The step of determining the driver's eye detection strategy based on the probability includes: The risk level of eye discomfort for the driver is determined based on the probability. If the risk level is Level 1 or Level 2, then eye detection is performed on the driver, and the driver's eye detection strategy is determined; wherein, the probability of Level 2 indication is less than the probability of Level 1 indication; If the risk level is level three, then no eye examination will be performed on the driver, wherein the probability of the level three indication is less than the probability of the level two indication.

12. The method according to claim 9, characterized in that, The step of performing route planning for the vehicle based on the eye detection strategy includes: Obtain the location information of the vehicle; If the location information indicates that the vehicle is in a non-parking area, the driving time of the vehicle from the non-parking area to the parking area is determined, and the driver is given an intervention prompt based on the driving time. The intervention prompt is used to remind the driver to take precautions to prevent eye discomfort. If the location information indicates that the vehicle is in a parking area, the driver will be given an immediate notification.

13. A driver status detection device, characterized in that, The device includes: The first acquisition module is used to acquire multiple frames of facial images of the driver within a first time period, and determine the driver state dataset based on the multiple frames of facial images. The first determining module is used to determine at least one of the driver's squeeze blink count, full blink count, incomplete blink count, and eye rubbing count based on the driver state dataset; The second determining module is used to determine the driver's eye state based on at least one of the number of squeeze blinks, the number of complete blinks, the number of incomplete blinks, and the number of eye rubbings.

14. A vehicle control device, characterized in that, The device includes: The second acquisition module is used to acquire the vehicle's historical travel data and the driver's historical eye data, and to determine the driver's total driving time in the first time period based on the historical travel data. The third determining module is used to determine the probability of the driver experiencing eye discomfort based on the total driving time and the historical eye data; the probability is determined by at least one of the driver's squeezing blinks, full blinks, incomplete blinks, and eye rubbing as indicated by the historical eye data. The control module is configured to determine the driver's eye detection strategy based on the probability, and execute the vehicle's trip planning based on the eye detection strategy; the trip planning includes vehicle control behavior when the vehicle is in a stopable area or a non-stopable area.

15. A vehicle, characterized in that, The vehicles include: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1-8 or 9-12.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1-8 or 9-12.

Images