Driving safety control method, electronic device, storage medium, and vehicle

Abstract

A driving safety control method, comprising: on the basis of eye state information of a driver and outside environment information of a vehicle, determining whether the eyes of the driver meet a field of view compensation trigger condition; and on the basis of the outside environment information of the vehicle, generating a driver field of view compensation image and providing same to the driver. The driving safety control method realizes the field of view compensation for the driver when the vision is limited or the field of view is insufficient, and effectively solves the problem of the vision being limited in complex scenarios such as light change, front windshield (5) obstruction, and severe weather, thereby improving the perception capability of the driver, reducing the risk of traffic accidents, and ensuring the driving safety of the driver. Further comprised are an electronic device (110), a storage medium, and a vehicle (100).

Description

Driving safety control methods, electronic devices, storage media and vehicles

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese patent application No. 202411031313.3, filed with the China National Intellectual Property Administration on July 30, 2024, entitled "Driving Safety Control Method, Electronic Device, Storage Medium and Vehicle", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to the field of vehicle technology, and in particular to a driving safety control method, electronic equipment, non-volatile readable storage medium, and vehicle. Background Technology

[0004] While driving, drivers often face situations where their vision is limited or insufficient. For example, drastic changes in light when entering or exiting tunnels can cause temporary blindness; debris adhering to the windshield can obstruct the driver's view; and sudden patches of fog on the road can also impair visibility. Furthermore, due to the limitations of the human eye's field of vision, drivers can easily overlook important traffic hazard information. All of these situations can seriously affect driving safety.

[0005] However, the relevant driving safety systems can only cope with single scenarios such as changes in light. They cannot effectively solve the problems of limited vision and insufficient driver visibility in other sudden severe weather conditions such as windshield obstruction and fog, which can easily lead to traffic accidents and fail to guarantee the driver's driving safety.

[0006] Public content

[0007] This disclosure aims to address at least one of the technical problems existing in the related art. To this end, one objective of this disclosure is to propose a driving safety control method that compensates for the driver's limited vision or insufficient field of vision, effectively solving the problem of limited vision in complex scenarios such as changing light, windshield obstruction, and inclement weather, thereby improving the driver's perception ability, reducing the risk of traffic accidents, and ensuring the driver's driving safety.

[0008] The second objective of this disclosure is to propose an electronic device.

[0009] The third objective of this disclosure is to propose a non-volatile readable storage medium.

[0010] The fourth objective of this disclosure is to propose a vehicle.

[0011] To achieve the above objectives, a driving safety control method according to a first aspect of this disclosure includes: determining that the driver's eyes meet the vision compensation triggering conditions based on driver's eye state information and external environment information; generating a driver vision compensation image based on the external environment information and providing it to the driver.

[0012] According to the driving safety control method of this disclosure, by acquiring the driver's eye state information and external environment information, the system monitors the driver's eye state and analyzes it in conjunction with the external environment to determine whether the driver's eyes meet the visual field compensation trigger condition. This means that the system can assess whether the driver's field of vision is limited or insufficient and determine whether visual field compensation is needed. When it is determined that the driver's field of vision meets the visual field compensation trigger condition, it indicates that the driver's field of vision is limited to a certain extent, and there is a situation of insufficient or limited field of vision. In this case, the system can generate a driver visual field compensation image based on the external environment information and provide it to the driver. In this way, the driver can effectively solve the problem of visual limitation in complex scenarios such as changes in light, windshield obstruction, and bad weather by obtaining the compensated image information. This allows the driver to obtain clearer and more comprehensive driving environment information, thereby improving the driver's perception ability, reducing the risk of traffic accidents, and ensuring the driver's driving safety.

[0013] In some embodiments, the eye state information includes pupil information; the visual field compensation triggering condition includes determining a pupil abnormality in the driver's eyes based on the pupil information and the environmental information.

[0014] In some embodiments, the pupil information includes pupil diameter and pupil diameter change rate; the pupil abnormality includes an abnormality in the illumination information corresponding to the pupil diameter and / or an abnormality in the illumination change gradient corresponding to the pupil diameter change rate; wherein, the illumination information and the illumination change gradient are obtained based on the environmental information.

[0015] In some embodiments, whether the pupil is abnormal is determined based on the output of a mapping function model of pupil diameter and illumination; wherein the mapping function model of pupil diameter and illumination takes the pupil diameter, the illumination information, the pupil diameter change rate, and the illumination change gradient as inputs.

[0016] In some embodiments, the eye state information includes iris image information; the visual field compensation triggering condition includes determining, based on the iris image information and the environmental information, that the driver's iris image does not contain target traffic element information; wherein the target traffic element information is obtained based on the environmental information.

[0017] In some embodiments, the iris image includes a background layer image and an upper layer image; the background layer image reflects the iris itself, and the upper layer image reflects external information detected by the human eye; the iris image does not include target traffic element information because the upper layer image does not include the target traffic element information.

[0018] In some embodiments, the driving safety control method further includes: validating the eye state information to obtain an eye state information verification value, and validating the environmental information to obtain an environmental information verification value; and obtaining functional requirement information of the whole vehicle for the driver's vision compensation function; the vision compensation triggering condition further includes that both the eye state information verification value and the environmental information verification value correspond to valid information and that the functional requirement information does not inhibit the driver's vision compensation function.

[0019] In some embodiments, generating a driver's field of vision compensation image based on the environmental information and providing it to the driver includes: generating head-up display information of the vehicle based on the environmental information and projecting it onto the head-up display to provide it to the driver.

[0020] In some embodiments, the head-up display information is enhanced display information, and the head-up display device is the windshield of the vehicle.

[0021] To achieve the above objectives, an electronic device according to a second aspect of this disclosure includes: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores a computer program executable by the at least one processor, and the at least one processor executes the computer program to implement the driving safety control method described in the above embodiment.

[0022] According to the electronic device of the present disclosure, the processor executes a computer program that implements the driving safety control method described in the above embodiments, which can realize vision compensation for the driver when vision is limited or field of vision is insufficient. This effectively solves the problem of limited vision in complex scenarios such as changes in light, windshield obstruction and bad weather, thereby improving the driver's perception ability, reducing the risk of traffic accidents, and ensuring the driver's driving safety.

[0023] To achieve the above objectives, a non-volatile readable storage medium according to a third aspect of this disclosure stores a computer program thereon, wherein the computer program, when executed, implements the driving safety control method described in the above embodiments.

[0024] According to the non-volatile readable storage medium of the present disclosure, by adopting the driving safety control method described in the above embodiments, vision compensation can be achieved when the driver's vision is limited or the field of vision is insufficient. This effectively solves the problem of limited vision in complex scenarios such as changes in light, windshield obstruction, and severe weather, thereby improving the driver's perception ability, reducing the risk of traffic accidents, and ensuring the driver's driving safety.

[0025] To achieve the above objectives, a vehicle according to a fourth aspect of this disclosure includes: an environmental information acquisition device for acquiring environmental information around the vehicle in the driving direction; a head-up display device for providing a driver's field-of-view compensation image to the driver; and a controller connected to the environmental information acquisition device and the head-up display device respectively, for acquiring the driver's eye state information and executing the driving safety control method described in the above embodiment.

[0026] According to the vehicle of this disclosure embodiment, the controller is connected to an environmental information acquisition device and a head-up display device respectively. It can obtain external environmental information collected by the environmental information acquisition device and execute the driving safety control method described in the above embodiment in combination with the obtained driver's eye state information. When it is determined that the driver's eyes meet the vision compensation trigger condition, it indicates that the driver's vision is limited to a certain extent, and there is insufficient or restricted vision. In this case, the head-up display device can be used to provide the driver with a driver vision compensation image generated based on the external environmental information. This effectively solves the problem of visual limitation in complex scenarios such as changes in light, windshield obstruction, and bad weather, thereby improving the driver's perception ability, reducing the risk of traffic accidents, and ensuring the driver's driving safety.

[0027] In some embodiments, the vehicle further includes an in-vehicle camera connected to the controller for collecting eye state information of the driver.

[0028] In some embodiments, the head-up display device includes an enhanced display device connected to the controller for projecting enhanced display information generated based on the environmental information onto the windshield of the vehicle.

[0029] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0030] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0031] Figure 1 is a schematic diagram of how a driver's eyes perceive a vehicle in a normal driving scenario in the relevant technology;

[0032] Figure 2 is a schematic diagram of how a driver's eyes perceive in a visually limited scenario in the relevant technology;

[0033] Figure 3 is a schematic diagram of a vehicle according to an embodiment of the present disclosure;

[0034] Figure 4 is a functional logic diagram of driving safety control according to an embodiment of the present disclosure;

[0035] Figure 5 is a flowchart of a driving safety control method according to an embodiment of the present disclosure;

[0036] Figure 6(a) is a schematic diagram of pupil constriction caused by the contraction of the circular muscle of the iris according to an embodiment of the present disclosure;

[0037] Figure 6(b) is a schematic diagram of pupil dilation caused by the contraction of the radial muscles of the iris according to an embodiment of the present disclosure;

[0038] Figure 7 is a functional logic diagram of a driving safety control method according to an embodiment of the present disclosure;

[0039] Figure 8 is a block diagram of an electronic device according to an embodiment of the present disclosure.

[0040] Reference numerals: Vehicle 100; Environmental information acquisition device 1; Controller 2; In-vehicle camera 3; Enhanced display device 4; Windshield 5; Head-up display device 6; Electronic equipment 110; Processor 111; Memory 112. Detailed Implementation

[0041] The embodiments of this disclosure are described in detail below, and the embodiments described with reference to the accompanying drawings are exemplary.

[0042] Figure 1 is a schematic diagram illustrating the driver's visual perception in a normal driving scenario in related technologies. As shown in Figure 1, under normal circumstances, the driver can see objects outside the vehicle (such as pedestrians, obstacles, and other vehicles) through the windshield. In the face of sudden dangerous situations, such as a pedestrian suddenly crossing the road or another vehicle suddenly changing lanes, the driver can react quickly, using steering and braking to avoid a collision or mitigate the consequences of an accident. Therefore, the driver's visual perception plays a crucial role in driving and directly affects driving safety.

[0043] Figure 2 is a schematic diagram illustrating how a driver's eyes perceive information in a visually limited scenario, as depicted in the related technology. As shown in Figure 2, during vehicle operation, if the windshield is obstructed by an object, or other environmental factors such as significant changes in ambient light levels cause visual limitations, the driver may be unable to fully perceive all information about the external environment. Under such visually limited conditions, the driver's reaction speed and accuracy may be affected, increasing uncertainty and risk in driving. For example, when the driver cannot clearly see pedestrians, vehicles, or obstacles ahead, they may be unable to take timely driving maneuvers to avoid potential dangers, such as emergency braking or evasive maneuvers. In such situations, both driver safety and vehicle stability may be affected, thereby increasing the risk of traffic accidents.

[0044] However, the relevant driving safety systems can only cope with single scenarios such as changes in light. They cannot effectively solve the problems of limited vision and insufficient driver visibility in other sudden severe weather conditions such as windshield obstruction and fog. This can easily lead to traffic accidents and fail to guarantee the driver's driving safety.

[0045] To address the aforementioned issues, this disclosure proposes a driving safety control method that compensates for the driver's limited vision or insufficient field of vision. This method effectively solves the problem of limited vision in complex scenarios such as changing light, windshield obstruction, and inclement weather, thereby enhancing the driver's perception, reducing the risk of traffic accidents, and ensuring the driver's driving safety.

[0046] To facilitate the explanation of the technical solution, the vehicle 100 of the present disclosure embodiment will be described below with reference to FIG3.

[0047] Figure 3 is a schematic diagram of a vehicle according to an embodiment of the present disclosure. As shown in Figure 3, the vehicle 100 includes: an environmental information acquisition device 1, a head-up display device 6, and a controller 2.

[0048] In some embodiments, the environmental information acquisition device 1 may include various types of sensors, such as cameras, lidar, millimeter-wave radar, and light sensors. These sensors can provide various types of environmental information, such as the size, position, speed, acceleration, light intensity, road conditions, and weather conditions of objects ahead, to help the system perceive the external environment more comprehensively.

[0049] In some embodiments, the environmental information acquisition device 1 can be installed at different locations on the vehicle 100 to achieve all-round monitoring of the external environment. For example, a camera can be installed above the windshield 5, on the side of the vehicle body, or at the four corners of the vehicle 100, while lidar and millimeter-wave radar can be installed on the roof of the vehicle 100, inside the bumper, or under the crash panel. The choice of installation location depends on the type, performance, and design considerations of the sensor, as well as the structure and appearance design of the vehicle 100, and is not specifically limited here.

[0050] In some embodiments, the specific number of environmental information acquisition devices 1 installed can be set according to various factors, such as vehicle model, size, safety requirements, and cost considerations. Depending on these factors, a single sensor or a combination of multiple sensors can be used to meet the needs of collecting and processing information about the external environment of the vehicle.

[0051] In some embodiments, the head-up display device 6 may be a display screen or a projection device for providing the driver with a driver's field of vision compensation image. Specifically, when the driver's field of vision is limited or insufficient, the system can generate a corresponding driver's field of vision compensation image based on environmental information and display it to the driver through the head-up display device 6 to help the driver obtain clearer and more comprehensive driving environment information.

[0052] In some embodiments, the controller 2 may be of various types for driving safety control, including but not limited to vehicle controller (VCU, Vehicle Control Unit), advanced driver assistance system controller (ADAS Controller), electronic control unit (ECU), or other dedicated controllers.

[0053] In some embodiments, the controller 2 is connected to the environmental information acquisition device 1 and the head-up display device 6 respectively, and is used to acquire the driver's eye state information and execute the driving safety control method described in the following embodiments. The controller 2 analyzes the driver's eye state information and environmental information to determine whether the driver's eyes meet the visual field compensation triggering conditions. If the conditions are met, a corresponding driver visual field compensation image is generated and provided to the driver, thereby realizing visual field compensation for the driver when vision is limited or the field of vision is insufficient.

[0054] In some embodiments, the controller 2 can also coordinate the work between other system modules and exchange data with external systems or other vehicle-mounted devices through the communication module to ensure the overall coordination and efficient operation of the system.

[0055] According to the vehicle 100 of this disclosure, the controller 2 is connected to the environmental information acquisition device 1 and the head-up display device 6 respectively. It can obtain the external environment information collected by the environmental information acquisition device 1 and combine it with the obtained driver's eye state information to execute the driving safety control method described in the above embodiment. When it is determined that the driver's eyes meet the vision compensation trigger condition, it indicates that the driver's vision is limited to a certain extent, and there is insufficient or limited vision. In this case, the head-up display device 6 can be used to provide the driver with a driver vision compensation image generated based on the external environment information. This effectively solves the problem of limited vision in complex scenarios such as changes in light, windshield obstruction, and bad weather, thereby improving the driver's perception ability, reducing the risk of traffic accidents, and ensuring the driver's driving safety.

[0056] In some embodiments, the vehicle 100 may further include an in-vehicle camera 3, which may be a high-definition camera or an infrared camera, or other types of cameras. The camera may be installed above the dashboard, above the steering column, or in the center of the roof to ensure accurate capture of the driver's eye position information.

[0057] In some embodiments, the in-vehicle camera 3 can be connected to the controller 2 via cable or wireless communication to collect the driver's eye state information. This eye state information may include pupil diameter and iris information. Specifically, after receiving the eye state information transmitted by the in-vehicle camera 3, the controller 2 can analyze and identify the eye state information using built-in algorithms and models to determine whether the driver's eye state is normal. When the controller 2 detects an abnormal eye state or a limited field of vision, it can meet the vision compensation trigger condition, generate a corresponding driver vision compensation image based on environmental information, and send it to the head-up display 6 for the driver to view.

[0058] In some embodiments, the in-vehicle camera 3 can be used not only to collect information about the driver's eye state, but also to identify the driver's facial expressions and fatigue state, as well as to monitor the driver's behavior.

[0059] In some embodiments, in addition to being monitored in real time by the in-vehicle camera 3, the driver's eye state information can also be collected by the camera of a wearable device or by the human-machine interface.

[0060] Wearable devices refer to smart devices worn by the driver, such as smart glasses, head-mounted displays, and smart helmets. These devices are equipped with cameras that can collect real-time information about the driver's eye status. The cameras on wearable devices can be positioned closer to the driver's eyes, providing more accurate eye status information. Wearable devices are mobile and flexible; they can follow the driver's head movements to maintain continuous eye monitoring, without being limited by the position of the in-vehicle camera 3. Through wireless communication technologies (such as Bluetooth and Wi-Fi), the wearable devices can transmit the collected eye status information to the vehicle 100's controller 2 for further analysis and processing.

[0061] In some embodiments, human-machine interface (HMI) technology is a technology that interacts directly with the human brain or nervous system, aiming to achieve more direct and efficient communication and control between humans and computers, devices, or other systems. In vehicle safety control systems, acquiring eye state information through an HMI can refer to using implanted or non-contact sensor devices to directly monitor or read signals from the driver's brain or optic nerve, thereby obtaining data about the eye state.

[0062] In some embodiments, the head-up display device 6 includes an enhanced display device 4, which may employ optical projection technology to convert digital information into visible light images and then project these images onto a semi-transparent display interface, so that the driver can simultaneously see the external environment and superimposed image information within normal line of sight.

[0063] In some embodiments, the enhanced display device 4 and the controller 2 can be connected via CAN bus, Ethernet or wireless communication technology to project enhanced display information generated based on environmental information onto the windshield 5 of the vehicle 100, enabling the driver to obtain compensated field of vision information in real time, thereby improving the driver's perception of the driving environment and reducing the risks caused by limited vision.

[0064] In some embodiments, in addition to the windshield 5, the augmented display device 4 can also project augmented display information onto other imageable objects, such as glasses. This application method enhances the system's flexibility and meets different driving needs and application scenarios.

[0065] Figure 4 is a functional logic diagram of driving safety control according to an embodiment of the present disclosure. As shown in Figure 4, when there are no obstructions on the windshield 5 of the vehicle 100 and no environmental factors affecting visibility occur, the driver can perceive target information with their own eyes. The in-vehicle camera 3 or wearable device is used to collect the driver's eye state information and send the information to the controller 2. At the same time, the environmental information collection device 1 can collect environmental information (such as target information) around the vehicle 100 in the driving direction and send the information to the controller 2. The controller 2 can analyze and process the eye state information received from the in-vehicle camera 3 to determine whether vision compensation is needed, that is, whether the driver's eyes are restricted. When the determination result is that the driver's eyes are not restricted and vision compensation is not needed, the target enhancement display function will be suppressed, and the controller 2 will not activate the enhancement display device 4.

[0066] When there are obstructions or environmental factors affecting the driver's vision on the windshield 5 of vehicle 100, the driver's eyes cannot fully perceive target information. At this time, the in-vehicle camera 3 or wearable device is used to collect the driver's eye state information and send the information to the controller 2. At the same time, the environmental information collection device 1 can collect environmental information (such as target information) around the vehicle 100 in the driving direction and send the information to the controller 2. The controller 2 can analyze and process the eye state information received from the in-vehicle camera 3 to determine whether vision compensation is needed, that is, whether the driver's vision is limited. When the determination result is that the driver's vision is limited and vision compensation is needed, the target enhancement display function will be activated. The controller 2 will send an activation command to the enhancement display device 4. The enhancement display device 4 can project the enhanced display information generated based on the environmental information onto the windshield 5 inside the vehicle using optical projection technology. The driver can perceive the enhanced display image on the windshield 5 and obtain the compensated vision information in real time, thereby improving the driver's perception of the driving environment and reducing the risks caused by visual limitations.

[0067] Based on the vehicle 100 described in the above embodiments, the driving safety control method according to an embodiment of the present disclosure is described below with reference to FIG5.

[0068] Figure 5 is a flowchart of a vehicle safety control method according to an embodiment of the present disclosure. As shown in Figure 5, the vehicle safety control method includes at least steps S1-S2, as follows:

[0069] S1, based on the driver's eye status information and external environment information, determine that the driver's eyes meet the visual field compensation triggering conditions.

[0070] In some embodiments, driver eye status information can be obtained in real time through sensors installed in the vehicle (such as in-vehicle cameras) or wearable devices worn by the driver (such as smart glasses). External environmental information can be obtained in real time through environmental information acquisition devices (such as cameras, lidar, millimeter-wave radar, and light sensors).

[0071] In some embodiments, external environmental information may include the area in front of and to the left and right of the vehicle's direction of travel, i.e., environmental information within the driver's field of vision while driving. The environmental information may include the position, speed, acceleration, light intensity, road conditions, and weather conditions of objects in front (pedestrians and obstacles, other vehicles, etc.).

[0072] In some embodiments, the vision compensation trigger condition can refer to a predefined standard or rule used to determine whether the driver's field of vision is limited or insufficient. Scenarios for which the vision compensation trigger condition applies may include situations such as detecting an obstacle ahead but the driver not reacting, drastic changes in light potentially causing temporary blindness, or the windshield being obstructed or fog affecting the driver's vision. When the vehicle encounters these situations while driving, the system will determine whether the driver's eyes meet the vision compensation trigger condition and take appropriate action to ensure driving safety.

[0073] S2 generates a driver's field of vision compensation image based on external environmental information and provides it to the driver.

[0074] In some embodiments, when the driver's eyes meet the vision compensation trigger condition, the target augmentation display function is activated, and the controller sends an activation command to the augmentation display device. The augmentation display device uses optical projection technology to project augmented display information generated based on environmental information onto the windshield inside the vehicle or onto other imageable objects, such as glasses. By perceiving the driver's vision compensation image on the windshield or other imageable objects, the driver obtains compensated vision information in real time, thereby improving the driver's perception of the driving environment and reducing the risks associated with visual impairment.

[0075] According to the driving safety control method of this disclosure, by acquiring the driver's eye state information and external environmental information, the system monitors the driver's eye state and analyzes it in conjunction with the external environmental conditions to determine whether the driver's eyes meet the visual field compensation trigger conditions. This means that the system can assess whether the driver's field of vision is limited or insufficient and determine whether visual field compensation is needed. When it is determined that the driver's field of vision meets the visual field compensation trigger conditions, it indicates that the driver's field of vision is limited to a certain extent, and there is a situation of insufficient or limited field of vision. In this case, the system can generate a driver visual field compensation image based on the external environmental information and provide it to the driver. In this way, the driver can effectively solve the problem of visual limitation in complex scenarios such as changes in light, windshield obstruction, and inclement weather by obtaining the compensated image information. This allows the driver to obtain clearer and more comprehensive driving environment information, thereby improving the driver's perception ability, reducing the risk of traffic accidents, and ensuring the driver's driving safety.

[0076] Figure 6(a) is a schematic diagram of pupil constriction caused by the contraction of the circular muscles of the iris according to an embodiment of the present disclosure, and Figure 6(b) is a schematic diagram of pupil dilation caused by the contraction of the radial muscles of the iris according to an embodiment of the present disclosure. As shown in Figures 6(a) and (b), the external area exposed to the human eye consists of three regions: the outer white area is the sclera, the middle gray area is the iris, and the inner black area is the pupil. The size of the pupil is controlled by two sets of muscles within the iris: the circular muscles (also known as the pupillary sphincter muscles) and the radial muscles (also known as the pupillary dilators). When the light is strong, the circular muscles contract, causing the pupil to constrict, thus reducing the amount of light entering the eye and protecting the retina from strong light stimulation. Figure 6(a) illustrates this process, with the pupil noticeably constricted. When the light is weak, the radial muscles contract, and the circular muscles relax, causing the pupil to dilate, thus increasing the amount of light entering the eye and enhancing vision. Figure 6(b) illustrates this process, with the pupil noticeably dilated.

[0077] In some embodiments, eye status information includes pupil information, which can be physiological data related to the driver's pupils, such as pupil size, pupil response speed to changes in light, and the degree of pupil dilation or constriction. The system can monitor and collect the driver's pupil information in real time via an in-vehicle camera or a camera on a wearable device.

[0078] In some embodiments, the vision compensation trigger condition includes determining pupil abnormalities in the driver's eyes based on pupil information and environmental information. Pupil abnormalities can refer to abnormal pupil responses under the influence of changes in light, fatigue, or other factors. The system can determine whether the pupils are abnormal based on preset standards and thresholds, combined with environmental information. For example, when entering or exiting a tunnel, if the driver experiences temporary blindness due to excessive changes in light intensity, the system may detect that the pupils are not constricting or dilating normally. In this case, a pupil abnormality can be determined, satisfying the vision compensation trigger condition. The controller can then generate a vision compensation image based on the environmental information. The compensation image is projected onto the windshield via an enhanced display device, ensuring that the driver can obtain clear visual information in a timely manner, such as road conditions, vehicles, and pedestrians ahead.

[0079] In some embodiments, pupil information may include pupil diameter and pupil diameter change rate. Pupil diameter reflects the size of the pupil and is the result of the pupil's automatic adjustment in response to changes in light. The pupil diameter can be determined by analyzing the pupil's diameter in an image captured by a camera using image processing techniques. The pupil diameter change rate represents the rate at which the pupil size changes over time, reflecting the pupil's sensitivity to changes in light. The pupil diameter change rate is obtained by continuously acquiring pupil diameter data, calculating the change in pupil diameter between two adjacent acquisition intervals, and then dividing by the time interval.

[0080] In some embodiments, pupillary abnormalities include abnormalities in the illumination information corresponding to pupil diameter and / or abnormalities in the illumination gradient corresponding to the rate of change of pupil diameter. That is, when only the illumination information corresponding to pupil diameter is abnormal, it can be determined that the driver's eyes have a pupillary abnormality. Alternatively, when only the illumination gradient corresponding to the rate of change of pupil diameter is abnormal, it can be determined that the driver's eyes have a pupillary abnormality. Alternatively, when both the illumination information corresponding to pupil diameter and the illumination gradient corresponding to the rate of change of pupil diameter are abnormal, it can also be determined that the driver's eyes have a pupillary abnormality.

[0081] The pupil diameter change rate refers to the rate at which the pupil diameter changes over time. By monitoring this rate, the system can determine the driver's response speed to changes in lighting conditions. Under normal circumstances, the pupil adjusts its size according to changes in ambient light to regulate the amount of light received by the eye, thereby maintaining appropriate visual comfort. The lighting gradient refers to the rate at which ambient light intensity changes over time. By monitoring the rate of change in ambient light intensity, the system can understand the lighting conditions in the driver's environment.

[0082] In some embodiments, illumination information and illumination change gradients are obtained based on environmental information, i.e., the illumination intensity and illumination changes of the surrounding environment can be acquired through an environmental information acquisition device (such as a photosensor). Illumination information reflects the current ambient light intensity, while the illumination change gradient represents the rate of change of light intensity. When the ambient light intensity is high, for example, when a vehicle travels through a long stretch of brightly lit road, monitoring and analyzing the pupil diameter can indicate a pupillary abnormality in the driver. Similarly, when the ambient light suddenly dims or brightens, such as when a vehicle enters or exits a tunnel, monitoring and analyzing the rate of change in pupil diameter can also indicate a pupillary abnormality in the driver.

[0083] In some embodiments, whether the pupil is abnormal is determined based on the output of a mapping function model between pupil diameter and illumination. This mapping function model takes pupil diameter, illumination information, the rate of change of pupil diameter, and the gradient of illumination change as inputs.

[0084] In some embodiments, the mapping function model can refer to a pre-established mathematical model or lookup table of illumination and pupil size. The lookup table can be fixed and established using data collected under laboratory conditions. For example, it can record changes in pupil diameter under different illumination conditions. Alternatively, it can be dynamically updated and maintained by collecting driver status data daily. By inputting information such as pupil diameter, illumination information, pupil diameter change rate, and illumination change gradient into the lookup table for analysis and comparison, when the pupil diameter or pupil diameter change rate exceeds a predefined threshold range, it is determined to be a pupil abnormality.

[0085] In some embodiments, the output of the model can be a binary signal, for example, an output of 0 indicates an abnormal pupil and an output of 1 indicates a normal pupil.

[0086] In some embodiments, the eye state information also includes iris image information. The iris image information can be an image of the driver's iris captured by an in-vehicle camera or a wearable device camera. The iris is a unique and stable biometric feature of the human eye. The iris image provides detailed information about the driver's gaze direction, helping to determine whether the driver's gaze contains information about target traffic elements.

[0087] In some embodiments, the vision compensation triggering condition includes determining, based on iris image information and environmental information, that the driver's iris image does not contain target traffic element information. The target traffic element information is obtained based on the environmental information.

[0088] In some embodiments, target traffic element information may refer to information about specific targets or objects in the environmental information that are directly related to traffic safety. Target traffic element information may include, but is not limited to, important traffic signs, road signs, pedestrians, obstacles, and other vehicles in the direction of travel.

[0089] In some embodiments, target traffic element information about the vehicle's driving direction is acquired by environmental information acquisition devices (such as cameras and radar) and transmitted to the controller. Iris image information of the driver's eyes is captured by an in-vehicle camera or a wearable device camera and transmitted to the controller. The controller then analyzes and compares the target traffic element information and the iris image information. Under normal driving conditions, the driver's line of sight should cover key surrounding traffic elements to ensure driving safety. Therefore, the iris image information should include a reflection of these traffic elements. For example, when the driver is looking ahead, the iris image captured by the camera should include image information of the traffic signs ahead. When the analysis shows that the iris image does not contain target traffic element information, it indicates that the driver's field of vision is limited, which meets the triggering conditions for vision compensation, triggering the vision compensation mechanism.

[0090] In some embodiments, the iris image includes a background layer image and an upper layer image. The background layer image reflects the shape, texture, and color of the iris itself, and is the basic structural information of the iris. The upper layer image reflects external information detected by the human eye, including target traffic elements in the surrounding environment, such as traffic signs, pedestrians, and vehicles.

[0091] In some embodiments, the absence of target traffic element information in the iris image means that the absence of target traffic element information in the upper layer image. Therefore, during iris image processing, the system analyzes the iris image using algorithms. The background layer image is mainly used to identify the driver's identity and eye features, while the upper layer image is used to determine whether the driver can see the target traffic element information of the surrounding environment.

[0092] In some embodiments, the driving safety control method further includes: validating eye state information to obtain an eye state information verification value, validating environmental information to obtain an environmental information verification value, and obtaining functional requirement information of the vehicle for the driver's vision compensation function.

[0093] Among them, the eye status information verification value and the environmental information verification value are values ​​generated after validating the corresponding information. They reflect the reliability of the verified eye status information and environmental information, ensuring that the acquired eye status information and environmental information are accurate and reliable for subsequent processing and analysis.

[0094] In some embodiments, validity verification may involve checking the integrity, accuracy, and legality of data. In this scenario, validity verification of eye state information and environmental information can be implemented using a state machine. A state machine is a method of implementing control logic that uses a series of algorithms or logic to check the validity of eye state information and environmental information, ensuring that it conforms to the expected format, range, or logical relationship. Therefore, a state machine can perform different operations or decisions based on different states of the input signal, thereby determining the validity of the information.

[0095] In some embodiments, the vehicle system may suppress the driver visibility compensation function based on the needs or limitations of certain specific situations. These situations may include low battery, system malfunction, or the driver manually disabling the assistance function. Therefore, obtaining functional requirement information helps the system determine whether it is necessary to suppress the driver visibility compensation function.

[0096] In some embodiments, obtaining the vehicle's functional requirements for the driver's field of vision compensation function can also be achieved through a state machine. The state machine can be used to monitor the system's state and the execution of various functions, and generate corresponding functional requirements based on set conditions, thereby limiting the driver's field of vision compensation function.

[0097] In some embodiments, the vision compensation triggering conditions also include eye state information verification values ​​and environmental information verification values, both of which correspond to valid information and functional requirement information indicating that the driver's vision compensation function should not be suppressed. This means that all three conditions must be met simultaneously; if any one condition is not met, the vision compensation triggering conditions cannot be met. For example, even if both the eye state information and environmental information verification values ​​are valid, if the functional requirement information indicates that the driver's vision compensation function needs to be suppressed, then the driver's vision compensation function will not be activated.

[0098] Figure 7 is a functional logic diagram of a driving safety control method according to an embodiment of the present disclosure. As shown in Figure 7, the environmental information acquisition device is used to collect external environmental information and send it to the environmental information processing module in the controller. After receiving the environmental information, the environmental information processing module analyzes and processes it, extracts the illumination information and illumination change gradient, and then transmits this information to the pupil anomaly confirmation module of the controller. Simultaneously, the environmental information processing module can also extract target traffic element information and then transmit this information to the iris anomaly confirmation module. When processing the environmental information, the environmental information processing module also needs to perform validity verification to obtain an environmental information verification value. If an abnormality is detected in the environmental information, such as a camera malfunction or the environmental information exceeding the normal range, the module will reflect the abnormal field in the environmental information verification value to alert subsequent processing modules to the abnormality.

[0099] In addition, an in-vehicle camera is used to collect the driver's eye state information and send it to the human eye image processing module in the controller. Upon receiving the driver's eye state information, the human eye image processing module analyzes and processes it, extracting the pupil diameter and pupil diameter change rate, and transmits this information to the pupil anomaly confirmation module. Simultaneously, for the iris region image information, this module can create multi-layered images to distinguish between the background image reflecting the iris itself and the upper layer image reflecting external information detected by the human eye. Then, the upper layer image information of the iris region is transmitted to the iris anomaly confirmation module.

[0100] Furthermore, the pupil anomaly confirmation module may include a light intensity-pupil relationship table, showing the pupil diameter under different light conditions. This table can be a fixed form or updated and maintained based on daily driver status information. The pupil anomaly confirmation module can obtain light intensity information and light intensity gradient from the environmental information processing module, and pupil diameter and pupil diameter change rate from the human eye image processing module, then compare them with the internal relationship table. If the pupil diameter or pupil diameter change rate exceeds a preset threshold range, it indicates that the driver's eyes have a pupil anomaly. In this case, the iris anomaly confirmation module will send pupil anomaly information to the visual impairment output module in the controller to remind the system to take appropriate measures to ensure driving safety.

[0101] Furthermore, the iris anomaly confirmation module can obtain target traffic element information from the environmental information processing module and the upper layer image of the iris from the human eye image processing module, and compare them. If the target traffic element information does not appear in the upper layer image of the iris, iris anomaly information is sent to the visually restricted output module. At the same time, when processing eye state information, the human eye image processing module also needs to perform validity verification on the eye state information to obtain an eye state information verification value.

[0102] Furthermore, the state machine can acquire environmental information verification values, eye state information verification values, and the vehicle's functional requirements for the driver's field of vision compensation function. When both the eye state information verification values ​​and the environmental information verification values ​​are valid, and the functional requirements do not inhibit the driver's field of vision compensation function, the state machine is activated, and the visually restricted output module can output the received pupil and iris abnormality information to the outside. When any one of the above three conditions is not met, the state machine's function is restricted, and the visually restricted output module cannot output the received pupil and iris abnormality information to the outside.

[0103] In some embodiments, generating a driver's field of vision compensation image based on environmental information and providing it to the driver includes: generating head-up display information of the vehicle based on environmental information and projecting it onto the head-up display to provide it to the driver.

[0104] In some embodiments, when the driver's eyes meet the vision compensation trigger condition, the controller can generate head-up display information suitable for the driver's viewing based on external environmental information. This information may include the road conditions ahead, traffic signs, the position and movement of various objects such as other vehicles and pedestrians, as well as environmental factors such as weather and lighting. The controller can then project this head-up display information onto the vehicle's head-up display so that the driver can intuitively access this information while driving. This allows the driver to drive the vehicle more safely and conveniently.

[0105] In some embodiments, the head-up display information is enhanced display information, and the head-up display device is the vehicle's windshield. Enhanced display information can refer to display information that has been processed, enhanced, or has additional content added to the original information. Such processing may include increasing contrast, adjusting colors, and overlaying navigation signs, to provide the driver with clearer and more comprehensive information about the driving environment.

[0106] In some embodiments, the windshield, serving as the projection plane for the head-up display, is transparent and offers a wide field of view, allowing the driver to clearly see the projected information without interfering with their normal line of sight. This layout design enables the driver to directly access crucial driving information without shifting their gaze, improving driving safety and convenience.

[0107] In addition, to increase the system's flexibility and meet different driving needs and application scenarios, the enhanced display information can also be projected onto other imageable objects, such as glasses.

[0108] An electronic device according to an embodiment of the present disclosure is described below with reference to FIG8.

[0109] FIG8 is a block diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG8, the electronic device 110 includes a memory 112 and at least one processor 111.

[0110] The processor 111 can be one processor 111, or it can be two processors 111, three processors 111, five processors 111, eight processors 111, ten processors 111, or more processors 111. These processors 111 can be general-purpose processors or special-purpose chips, depending on the design and purpose of the electronic device 110.

[0111] In some embodiments, memory 112 may be used to store computer programs and other necessary data. This memory 112 may include RAM (random access memory) and ROM (read-only memory), etc. The computer program is stored in memory 112 and awaits execution by processor 111.

[0112] In some embodiments, the memory 112 is communicatively connected to at least one processor 111, and the memory 112 stores a computer program that can be executed by at least one processor 111. When the at least one processor 111 executes the computer program, it implements the driving safety control method described in the above embodiments.

[0113] According to the electronic device 110 of this disclosure, the processor 111 executes a computer program that implements the driving safety control method described in the above embodiments, which can realize vision compensation for the driver when vision is limited or field of vision is insufficient. This effectively solves the problem of limited vision in complex scenarios such as changes in light, windshield obstruction and bad weather, thereby improving the driver's perception ability, reducing the risk of traffic accidents, and ensuring the driver's driving safety.

[0114] This disclosure also proposes a non-volatile readable storage medium storing a computer program, which, when executed by processor 111, implements the driving safety control method described in the above embodiments. The specific implementation process of the driving safety control method can be referred to the description in the above embodiments.

[0115] The computer-readable storage medium disclosed herein may include, but is not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other optical and magnetic storage media, which will not be described in detail here.

[0116] According to the non-volatile readable storage medium of the present disclosure, by adopting the driving safety control method described in the above embodiments, vision compensation can be achieved when the driver's vision is limited or the field of vision is insufficient. This effectively solves the problem of limited vision in complex scenarios such as changes in light, windshield obstruction, and severe weather, thereby improving the driver's perception ability, reducing the risk of traffic accidents, and ensuring the driver's driving safety.

[0117] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example.

[0118] Although embodiments of this disclosure have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this disclosure, the scope of which is defined by the claims and their equivalents.

Claims

1. A method of controlling vehicle safety, wherein, comprising: determining that the driver's eyes meet a field of view compensation trigger condition according to eye state information of the driver and environment information outside the vehicle; and generating a driver field of view compensation image according to the environment information and providing the driver field of view compensation image to the driver.

2. The driving safety control method according to claim 1, wherein the eye state information comprises pupil information; the field of view compensation trigger condition comprises determining that the pupil of the driver's eyes is abnormal according to the pupil information and the environment information.

3. The driving safety control method according to claim 2, wherein the pupil information comprises a pupil diameter and a pupil diameter change rate; the pupil abnormality comprises that the pupil diameter corresponding to illumination information is abnormal and / or the pupil diameter change rate corresponding to an illumination change gradient is abnormal; wherein the illumination information and the illumination change gradient are obtained according to the environment information.

4. The driving safety control method according to claim 3, wherein whether the pupil is abnormal is determined according to the output result of a mapping function model of the pupil diameter and the illumination; wherein the mapping function model of the pupil diameter and the illumination takes the pupil diameter, the illumination information, the pupil diameter change rate and the illumination change gradient as input.

5. The driving safety control method according to any one of claims 1-4, wherein the eye state information comprises iris image information; the field of view compensation trigger condition comprises determining that the target traffic element information is not included in the driver's iris image according to the iris image information and the environment information; wherein the target traffic element information is obtained according to the environment information.

6. The driving safety control method according to claim 5, wherein the iris image comprises a background layer image and an upper layer image, the background layer image reflects the iris itself, and the upper layer image reflects the external information detected by the human eye; the target traffic element information is not included in the upper layer image.

7. The vehicle safety control method according to any one of claims 1-6, wherein, The driving safety control method further comprises: validity checking the eye state information to obtain an eye state information check value, and validity checking the environment information to obtain an environment information check value; and obtaining functional requirement information of the whole vehicle for the driver field of view compensation function; the field of view compensation trigger condition further comprises that the eye state information check value and the environment information check value both correspond to valid information and the functional requirement information corresponds to not inhibiting the driver field of view compensation function.

8. The vehicle safety control method according to any one of claims 1-7, wherein, Generating a driver field of view compensation image according to the environment information to provide to the driver comprises: generating head-up display information of the vehicle according to the environment information and projecting it onto a head-up display piece to provide to the driver.

9. The vehicle safety control method according to claim 8, wherein The head-up display information is augmented display information, and the head-up display piece is the front windshield of the vehicle.

10. An electronic device (110), wherein comprising: at least one processor (111); and a memory (112) in communication with the at least one processor (111); The memory (112) stores a computer program executable by the at least one processor (111), and the at least one processor (111) implements the driving safety control method of any one of claims 1-9 when executing the computer program.

11. A non-transitory readable storage medium having stored thereon a computer program, wherein, The computer program is executed to implement the driving safety control method of any one of claims 1-9.

12. A vehicle (100), wherein Comprise: An environmental information acquisition device (1) for acquiring external environment information of the vehicle; A head-up display device (6) for providing a driver's field of view compensation image to the driver; And A controller (2) connected with the environmental information acquisition device (1) and the head-up display device (6) respectively, for acquiring eye state information of the driver, and executing the driving safety control method of any one of claims 1-9.

13. The vehicle (100) according to claim 12, wherein The vehicle (100) further comprises: An in-vehicle camera (3) connected with the controller (2) for acquiring the eye state information of the driver.

14. The vehicle (100) according to claim 12 or 13, wherein The head-up display device (6) comprises: An augmented display device (4) connected with the controller (2) for projecting augmented display information generated according to the environmental information to the front windshield (5) of the vehicle (100).

Images